# US 10,771,550 B2 · Data storage system with redundant internal networks > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,771,550 B2 - Document ID: US10771550B2 - Status: Granted - Kind: B2 - Application number: 15/392,835 - Filed: 2016-12-28 - Published: 2018-06-28 - Granted: 2020-09-08 - Earliest claimed: 2016-12-28 - Assignees: Amazon Technologies, Inc. - Applicants: Amazon Technologies, Inc. - Inventors: Norbert P. Kusters; Nachiappan Arumugam; Christopher Nathan Watson; Marc John Brooker; David R. Richardson; Danny Wei; John Luther Guthrie, II - Web page: https://patentgrep.com/patent/US10771550B2 ## CPC - H04L 67/1095: Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes - G06F 3/0614: {Improving the reliability of storage systems} - G06F 3/0635: {by changing the path, e.g. traffic rerouting, path reconfiguration} - G06F 3/0647: {Migration mechanisms} - G06F 3/065: {Replication mechanisms} - G06F 3/067: {Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]} - G06F 3/0685: {Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays} - G06F 11/2005: {using redundant communication controllers} - H04L 61/2503: Translation of Internet protocol [IP] addresses - H04L 67/1097: for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] - H04L 67/2885: Hierarchically arranged intermediate devices, e.g. for hierarchical caching - H04L 67/568: Storing data temporarily at an intermediate stage, e.g. caching - G06F 11/1076: {Parity data used in redundant arrays of independent storages, e.g. in RAID systems} - G06F 11/1662: {the resynchronized component or unit being a persistent storage device (re-synchronization of failed mirror storage G06F11/2082; rebuild or reconstruction of parity RAID storage G06F11/1008)} - G06F 11/2015: {Redundant power supplies (power supply failure G06F1/30)} - G06F 11/2082: {Data synchronisation} - G06F 11/3409: {for performance assessment} - H03M 13/154: {Error and erasure correction, e.g. by using the error and erasure locator or Forney polynomial} - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H03M: CODING; DECODING; CODE CONVERSION IN GENERAL (using fluidic means F15C4/00; optical analogue/digital converters G02F7/00; coding, decoding or code conversion, specially adapted for particular applications, see the relevant subclasses, e.g. G01D, G01R, G06F, G06T, G09G, G10L, G11B, G11C, H04B, H04L, H04M, H04N; ciphering or deciphering for cryptography or other purposes involving the need for secrecy G09C) ## Abstract A data storage system includes a rack, multiple head nodes, multiple data storage sleds, and at least two networking devices. The at least two network devices are configured to implement at least two redundant networks within the data storage system. Also, each of the head nodes is assigned at least two network addresses for communication with the data storage sleds of the data storage system via the at least two networking devices. The data storage sleds each include multiple mass storage devices and a sled controller that is configured to couple with the at least two network switches. In some embodiments, the data storage system further includes redundant power systems within a rack in which the head nodes, the data storage sleds, and the at least two networking devices are mounted. ## Claims ### Claim 1 A data storage system, comprising: a plurality of head nodes; a plurality of data storage sleds; and at least two networking devices, wherein the at least two networking devices are configured to implement at least two redundant networks within the data storage system, wherein to implement the at least two redundant networks: each respective head node is coupled to each of the plurality of data storage sleds via a first one of the at least two networking devices; each respective head node is also coupled to each of the plurality of data storage sleds via a second one of the at least two networking devices; and each respective head node is assigned: a first network address for communicating with the plurality of data storage sleds via a path through a first one of the at least two networking devices; at least one additional network address for communicating with the plurality of data storage sleds via at least one additional path through at least one additional networking device of the at least two networking devices; a first external network IP address for communicating with devices external to the data storage system via a path through the first one of the at least two networking devices; and at least one additional external network IP address for communicating with devices external to the data storage system via a path through at least one additional networking device of the at least two networking devices; wherein each of the at least two redundant networks connects respective ones of the head nodes to the respective data storage sleds via a single network hop; and wherein the at least two redundant networks provide redundant communication paths within the data storage system between the respective ones of the head nodes and each of the plurality of data storage sleds such that a failure of one of the at least two networking devices does not prevent communication within the data storage system between the respective ones of the head nodes and the plurality of data storage sleds. ### Claim 2 (depends on 1) The data storage system of claim 1 further comprising: at least two connectors for coupling the data storage system to at least two electrical power sources; and at least two power distribution systems within the data storage system configured to supply redundant power from the at least two electrical power sources to each of the head nodes and to each of the data storage sleds. ### Claim 3 (depends on 1) The data storage system of claim 1, wherein: each respective head node is coupled to each of the plurality of data storage sleds via a single network hop at the first one of the at least two networking devices; and each respective head node is also coupled to each of the plurality of data storage sleds via a single network hop at the second one of the at least two networking devices. ### Claim 4 (depends on 1) The data storage system of claim 1, wherein the at least two network addresses for communicating with the plurality of data storage sleds are local addresses that are local to the data storage system. ### Claim 5 (depends on 1) The data storage system of claim 1, wherein a quantity of head nodes or a quantity of data storage sleds is adjustable within a rack. ### Claim 6 A data storage system, comprising: a head node comprising at least four network interfaces, wherein at least two of the network interfaces are configured to implement at least two redundant networks within the data storage system via different respective ones of at least two networking devices of the data storage system, and wherein another at least two of the network interfaces of are configured to enable communications between the head node and external clients via different respective ones of the at least two networking devices of the data storage system; wherein to implement the at least two redundant networks within the data storage system, the at least two network interfaces of the head node are configured to: couple to respective ones a plurality of data storage sleds via a first networking device; and couple to the respective ones of the plurality of data storage sleds via a second networking device, wherein the head node is assigned: a first network address for communicating with the plurality of data storage sleds via a first path through the first networking device; and at least one additional network address for communicating with the plurality of data storage sleds via a second path through the second networking device; wherein each of the at least two redundant networks within the data storage system connects the head node to the respective data storage sleds via a single network hop; wherein the at least two redundant networks provide redundant communication paths within the data storage system between the head node and each of the plurality of data storage sleds such that a failure of the first networking device or the second networking device does not prevent communication within the data storage system between the head node and the plurality of data storage sleds; wherein the head node is further assigned at least two additional network addresses for communicating with devices external to the data storage system via the first or second networking devices; and wherein the other at least two network interfaces of the head node are configured to: enable communications between the head node and external clients via the first networking device using a first one of the at least two additional network addresses; and enable communications between the head node and external client via the second networking device using a second one of the at least two additional network addresses. ### Claim 7 (depends on 6) The data storage system of claim 6, wherein the first network address for communicating with the plurality of data storage sleds and the at least one additional network address for communicating with the plurality of data storage sleds are local addresses that are local to the data storage system. ### Claim 8 (depends on 6) The data storage system of claim 6, wherein the head node is configured to mount in a rack with one or more additional head nodes and the plurality of data storage sleds. ### Claim 9 (depends on 6) The data storage system of claim 6, wherein the head node is configured to receive electrical power from at least two electrical power distribution systems of the data storage system such that the head node is supplied with redundant electrical power. ### Claim 10 (depends on 6) The data storage system of claim 6, wherein the head node comprises a storage drive, wherein based, at least in part, on a request to write data, the head node is configured to: write the data to the storage drive of the head node; and cause the data to be replicated to another head node in the data storage system via one of the at least two redundant networks within the data storage system. ### Claim 11 (depends on 10) The data storage system of claim 10, wherein the storage drive of the head node is a log-structured storage drive, and wherein the other head node comprises a log-structured storage drive. ### Claim 12 (depends on 11) The data storage system of claim 11 wherein the head node is further configured to cause the data to be transmitted to a sub-set of the plurality of data storage sleds via the at least one redundant network or via the other at least one redundant network within the data storage system, wherein respective portions of the data transmitted to the sub-set of the plurality of data storage sleds are stored in respective portions of storage devices included in the sub-set of the plurality of data storage sleds. ### Claim 13 (depends on 12) The data storage system of claim 12, wherein the head node is configured to asynchronously cause the data to be transmitted to the sub-set of the plurality of data storage sleds relative to said cause the data to be transmitted to the other head node for replication. ### Claim 14 A data storage system comprising: a data storage sled comprising at least two network interfaces configured to implement at least two redundant networks within the data storage system, wherein to implement the at least two redundant networks, the at least two network interfaces of the data storage sled are configured to: couple to a particular head node via a first a networking device; and couple to the same particular head node via a second networking device, wherein the data storage sled is assigned: a first network address for communicating with the particular head node or one or more additional head nodes via the first networking device; and a second network address for communicating with the particular head node or the one or more additional head nodes via the second networking device; wherein each of the at least two redundant networks connects the particular head node to the data storage sled via a single network hop; and wherein the at least two redundant networks provide redundant communication paths within the data storage system between the particular head node and the data storage sled, via the at least two network interfaces, such that a failure of the first networking device or the second networking device does not prevent communication within the data storage system between the particular head node and the data storage sled via at least one of the at least two network interfaces, wherein the particular head node is assigned at least two additional network address for communicating with devices external to the data storage system via respective ones of the first and second networking devices. ### Claim 15 (depends on 14) The data storage system of claim 14, wherein the first and second network addresses are local network addresses local to the data storage system. ### Claim 16 (depends on 14) The data storage system of claim 14, wherein the data storage sled is configured to receive electrical power from at least two electrical power distribution systems of the data storage system such that the data storage sled is supplied with redundant electrical power. ### Claim 17 (depends on 14) The data storage system of claim 14, wherein the data storage sled comprises: a plurality of solid state storage devices; and a sled controller, wherein the data storage sled is configured to mount on a shelf of a rack of the data storage system along with one or more additional data storage sleds of the data storage system. ### Claim 18 (depends on 14) The data storage system of claim 14, wherein the data storage sled comprises a plurality of mass storage devices configured to store at least one erasure encoded portion of a volume hosted by the data storage system.