# US 2017/0286243 A1 · Failover Handling In Modular Switched Fabric For Data Storage Systems > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2017/0286243 A1 - Document ID: US20170286243A1 - Status: Published application - Kind: A1 - Application number: 15/625,216 - Filed: 2017-06-16 - Published: 2017-10-05 - Earliest claimed: 2014-06-23 - Assignees: Liqid Inc. - Applicants: Liqid Inc. - Inventors: Jason Breakstone; Christopher R. Long; James Scott Cannata - Web page: https://patentgrep.com/patent/US20170286243A1 ## CPC - G06F 11/2028: {eliminating a faulty processor or activating a spare} - G06F 3/0607: {by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device} - G06F 3/0635: {by changing the path, e.g. traffic rerouting, path reconfiguration} - G06F 3/0685: {Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays} - G06F 11/2007: {using redundant communication media} - G06F 11/2012: {and using different communication protocols} - G06F 11/2017: {where memory access, memory control or I/O control functionality is redundant (redundant communication control functionality G06F11/2005; redundant storage control functionality G06F11/2089)} - G06F 11/2038: {with a single idle spare processing component} - G06F 11/2046: {where the redundant components share persistent storage (G06F11/2043 takes precedence)} - G06F 11/2053: {where persistent mass storage functionality or persistent mass storage control functionality is redundant (error detection or correction in information storage based on relative movement between record carrier and transducer G11B20/18)} - G06F 11/2069: {Management of state, configuration or failover} - G06F 13/4022: {using switching circuits, e.g. switching matrix, connection or expansion network (G06F13/4009 takes precedence)} - G06F 13/4068: {Electrical coupling} - G06F 13/4234: {being a memory bus} - G06F 13/4282: {on a serial bus, e.g. I2C bus, SPI bus (on daisy chain buses G06F13/4247)} - H04L 49/356: for storage area networks - H04L 67/1097: for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] - H04L 69/40: for recovering from a failure of a protocol instance or entity, e.g. service redundancy protocols, protocol state redundancy or protocol service redirection (management of faults, events, alarms or notifications in data switching networks H04L41/06) - G06F 2201/85: Active fault masking without idle spares - G06F 2213/0026: PCI express - Y02D 10/00: Energy efficient computing, e.g. low power processors, power management or thermal management - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - Y02D: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE ## Abstract Data storage systems are provided that include network interfaces each configured to receive storage operations issued by one or more host systems for handling by the data storage system, and transfer the storage operations over a Peripheral Component Interconnect Express (PCIe) fabric. The data storage systems also include processors configured to receive the storage operations over the PCIe fabric, process properties of the storage operations against allocation information of a shared PCIe address space to determine target processors to handle the storage operations, and selectively transfer ones of the storage operations to the target processors for handling of the storage operations with associated storage drives coupled over the PCIe fabric. The data storage systems also identifie failed ones of the processors and responsively initiate backup ones of the processors to handle portions of the shared PCIe address space handled by the failed ones of the processors. ## Claims ### Claim 1 A data storage system, comprising: a plurality of network interfaces each configured to receive storage operations issued by one or more host systems for handling by the data storage system, and transfer the storage operations over a Peripheral Component Interconnect Express (PCIe) fabric; a plurality of processors configured to receive the storage operations over the PCIe fabric, process properties of the storage operations against allocation information of a shared PCIe address space to determine target processors to handle the storage operations, and selectively transfer ones of the storage operations to the target processors for handling of the storage operations with associated storage drives coupled over the PCIe fabric; and a monitoring system configured to identify failed ones of the processors and responsively initiate backup ones of the processors to handle portions of the shared PCIe address space handled by the failed ones of the processors. ### Claim 2 (depends on 1) The data storage system of claim 1, comprising: the plurality of processors configured to maintain failover tables comprising indications of the allocation information of the shared PCIe address space among the plurality of processors. ### Claim 3 (depends on 1) The data storage system of claim 1, comprising: the plurality of processors comprising the monitoring system, and configured to periodically poll one or more monitored processors to identify failed ones of the processors. ### Claim 4 (depends on 1) The data storage system of claim 1, comprising: a control system configured to subdivide the shared PCIe address space to allocate a plurality of storage drives among individual processors, wherein each subdivision of the shared PCIe address space corresponds to an associated subset of the plurality of storage drives. ### Claim 5 (depends on 1) The data storage system of claim 1, comprising: a control system configured to assign each of a plurality of storage drives an associated unique identifier and report the associated unique identifiers to the processors for handling of the storage operations over the PCIe fabric. ### Claim 6 (depends on 1) The data storage system of claim 1, comprising: a control system configured to initialize a plurality of storage drives for use on the PCIe fabric by the plurality of processors; the control system configured to assign each of the plurality of storage drives an associated unique identifier using at least a sideband communication link distinct from the PCIe fabric, the sideband communication link communicatively coupling the control system and the plurality of storage drives; and the control system configured to report the associated unique identifiers to the plurality of processors. ### Claim 7 (depends on 1) The data storage system of claim 1, comprising: responsive to a failed processor, a monitoring processor configured to inherit network connection states from the failed processor and accept ones of the storage operations directed to a network interface initially assigned to the failed processor. ### Claim 8 (depends on 1) The data storage system of claim 1, comprising: responsive to a failed processor, a monitoring processor configured to initiate a rediscovery process over the PCIe fabric with one or more storage drives formerly managed by the failed processor to transfer management to the monitoring processor. ### Claim 9 (depends on 1) The data storage system of claim 1, comprising: responsive to detecting the failed ones of the processors, the monitoring system configured to restart the failed ones of the processors. ### Claim 10 A method of operating a data storage system, the method comprising: in a plurality of network interfaces, receiving storage operations issued by one or more host systems for handling by the data storage system, and transferring the storage operations over a Peripheral Component Interconnect Express (PCIe) fabric; in a plurality of processors, receiving the storage operations over the PCIe fabric, processing properties of the storage operations against allocation information of a shared PCIe address space to determine target processors to handle the storage operations, and selectively transferring ones of the storage operations to the target processors for handling of the storage operations with associated storage drives coupled over the PCIe fabric; and in a monitoring system, identifying failed ones of the processors and responsively initiating backup ones of the processors to handle portions of the shared PCIe address space handled by the failed ones of the processors. ### Claim 11 (depends on 10) The method of claim 10, further comprising: in the plurality of processors, maintaining failover tables comprising indications of the allocation information of the shared PCIe address space among the plurality of processors. ### Claim 12 (depends on 10) The method of claim 10, further comprising: the plurality of processors, periodically polling one or more monitored processors to identify failed ones of the processors. ### Claim 13 (depends on 10) The method of claim 10, further comprising: in a control system, subdividing the shared PCIe address space to allocate a plurality of storage drives among individual processors, wherein each subdivision of the shared PCIe address space corresponds to an associated subset of the plurality of storage drives. ### Claim 14 (depends on 10) The method of claim 10, further comprising: in a control system, assigning each of a plurality of storage drives an associated unique identifier, and reporting the associated unique identifiers to the processors for handling of the storage operations over the PCIe fabric. ### Claim 15 (depends on 10) The method of claim 10, further comprising: in a control system, initializing a plurality of storage drives for use on the PCIe fabric by the processors, and assigning each of the plurality of storage drives an associated unique identifier using at least a sideband communication link distinct from the PCIe fabric, the sideband communication link communicatively coupling the control system and the plurality of storage drives; and in the control system, reporting the associated unique identifiers to the plurality of processors. ### Claim 16 (depends on 10) The method of claim 10, further comprising: responsive to a failed processor, in a monitoring processor, inheriting network connection states from the failed processor and accepting ones of the storage operations directed to a network interface initially assigned to the failed processor. ### Claim 17 (depends on 10) The method of claim 10, further comprising: responsive to a failed processor, in a monitoring processor, initiating a rediscovery process over the PCIe fabric with one or more storage drives formerly managed by the failed processor to transfer management to the monitoring processor. ### Claim 18 (depends on 10) The method of claim 10, further comprising: responsive to detecting the failed ones of the processors, restarting the failed ones of the processors. ### Claim 19 A modular data storage system, comprising: a plurality of network modules each configured to receive storage operations issued by one or more host systems for handling by the data storage system, and transfer the storage operations over a Peripheral Component Interconnect Express (PCIe) fabric; a plurality of processing modules each comprising one or more processors configured to receive the storage operations over the PCIe fabric, process properties of the storage operations against allocation information of a shared PCIe address space to determine target processors to handle the storage operations, and selectively transfer ones of the storage operations to the target processors for handling of the storage operations with associated storage drives coupled over the PCIe fabric; a plurality of storage modules each comprising a plurality of storage drives communicatively coupled to the PCIe fabric, and monitoring ones of the processors configured to identify failed ones of the processors and responsively initiate backup ones of the processors to handle portions of the shared PCIe address space formerly handled by the failed ones of the processors. ### Claim 20 (depends on 19) The modular data storage system of claim 19, comprising: responsive to a failed processor, a monitoring processor configured to: inherit network connection states from the failed processor; accept ones of the storage operations directed to a network interface initially assigned to the failed processor; and initiate a rediscovery process over the PCIe fabric with one or more storage drives formerly managed by the failed processor to transfer management to the monitoring processor.