# US 2017/0288701 A1 · Restoration of Erasure-Coded Data Via Data Shuttle in Distributed Storage System > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2017/0288701 A1 - Document ID: US20170288701A1 - Status: Published application - Kind: A1 - Application number: 15/090,924 - Filed: 2016-04-05 - Published: 2017-10-05 - Earliest claimed: 2016-04-05 - Assignees: NetApp, Inc. - Applicants: NetApp, Inc. - Inventors: David Slik; Ronnie Lon Hei Chan; Vishnu Vardhan Chandra Kumaran - Web page: https://patentgrep.com/patent/US20170288701A1 ## CPC - G06F 11/1076: {Parity data used in redundant arrays of independent storages, e.g. in RAID systems} - H03M 13/373: {with erasure correction and erasure determination, e.g. for packet loss recovery or setting of erasures for the decoding of Reed-Solomon codes} - H03M 13/3761: {using code combining, i.e. using combining of codeword portions which may have been transmitted separately, e.g. Digital Fountain codes, Raptor codes or Luby Transform [LT] codes} - H03M 13/154: {Error and erasure correction, e.g. by using the error and erasure locator or Forney polynomial} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H03M: CODING; DECODING; CODE CONVERSION IN GENERAL (using fluidic means F15C4/00; optical analogue/digital converters G02F7/00; coding, decoding or code conversion, specially adapted for particular applications, see the relevant subclasses, e.g. G01D, G01R, G06F, G06T, G09G, G10L, G11B, G11C, H04B, H04L, H04M, H04N; ciphering or deciphering for cryptography or other purposes involving the need for secrecy G09C) ## Abstract Embodiments use data shuttle devices to restore erasure-coded data in a distributed storage environment. In some embodiments, a first data shuttle is communicatively coupled to a first node of the storage environment. On the data shuttle, first restoration data is generated from a first erasure-coded data portion stored on the first node. The first data shuttle or a second data shuttle is communicatively coupled to a second node of the storage environment. On the data shuttle at the second node, second restoration data is generated from a second erasure-coded data portion stored on the second node. Subsequent to transporting the first or second data shuttle from at least one of the other nodes to a third node, a third erasure-coded data portion is restored at the third node. The third erasure-coded data portion is generated via an erasure-coding process from one or more of the first or second restoration data. ## Claims ### Claim 1 A method comprising: detecting that a first data shuttle device is communicatively coupled to a first node of a distributed storage environment comprising the first node and a second node; generating, from a first erasure-coded data portion stored on the first node, first restoration data on the first data shuttle device responsive to detecting that the first data shuttle device is communicatively coupled to the first node; detecting that the first data shuttle device or a second data shuttle device is communicatively coupled to the second node; generating, from a second erasure-coded data portion stored on the second node, second restoration data on the first or second data shuttle device communicatively coupled to the second node, wherein the second restoration data is generated responsive to detecting that the first or second data shuttle device is communicatively coupled to the second node; and restoring a third erasure-coded data portion at a third node of the distributed storage environment subsequent to physically transporting at least one of the first or second data shuttle devices from at least one of the first or second nodes to a location of the third node, wherein the third erasure-coded data portion is generated via an erasure-coding process from at least one of the first restoration data stored on the first or second data shuttle device or the second restoration data stored on the first or second data shuttle device. ### Claim 2 (depends on 1) The method of claim 1, wherein generating and storing the first restoration data comprises copying the first erasure-coded data portion to the first data shuttle device; wherein the first data shuttle device is detected as being communicatively coupled to the second node subsequent to the first restoration data being generated and the first data shuttle device being decoupled from the first node; wherein the second restoration data comprises the third erasure-coded data portion and is generated on the first data shuttle device from the first erasure-coded data portion stored on the first data shuttle device and the second erasure-coded data portion stored on the second node; wherein restoring the third erasure-coded data portion at the third node comprises copying the third erasure-coded data portion from the first data shuttle device to the third node or installing the first data shuttle device as the third node after physically transporting the first data shuttle device from the second node to the location of the third node. ### Claim 3 (depends on 2) The method of claim 2, wherein generating and storing the second restoration data comprises: copying the first erasure-coded data portion from the first data shuttle device to a memory of the second node; generating the third erasure-coded data portion in the memory of the second node from the first erasure-coded data portion copied from the first data shuttle device and the second erasure-coded data portion stored at the second node; and replacing the first erasure-coded data portion stored on the first data shuttle device with the third erasure-coded data portion generated in the memory of the second node. ### Claim 4 (depends on 1) The method of claim 1, further comprising, prior to generating the first restoration data and the second restoration data: determining that the third node has failed; identifying the first node and the second node as respectively storing the first erasure-coded data portion and the second erasure-coded data portion; determining that the third node is to be restored using the at least one of the first data shuttle device or the second data shuttle device; and configuring the identified first and second nodes and the at least one of the first data shuttle device or the second data shuttle device for restoring the third erasure-coded data portion using the first erasure-coded data portion and the second erasure-coded data portion. ### Claim 5 (depends on 4) The method of claim 4, wherein configuring the identified first and second nodes and the at least one of the first data shuttle device or the second data shuttle device for restoring the third erasure-coded data portion comprises: identifying at least one device identifier for the at least one of the first data shuttle device or the second data shuttle device; and instructing the first node and the second node to respectively generate the first restoration data and the second restoration data in response to detecting a device having the at least one device identifier. ### Claim 6 (depends on 4) The method of claim 4, wherein determining that the third node is to be restored using the at least one of the first data shuttle device or the second data shuttle device comprises determining that an amount of data traffic generated by transmitting at least one of the first erasure-coded data portion or the second erasure-coded data portion via a data network to the third node exceeds a threshold amount of data traffic. ### Claim 7 (depends on 1) The method of claim 1, wherein at least one of the first or second data shuttle devices comprises or is included in a server and wherein restoring the third erasure-coded data portion at the third node comprises installing the server as the third node. ### Claim 8 (depends on 1) The method of claim 1, wherein at least one of the first or second data shuttle devices attaches to a server and wherein restoring the third erasure-coded data portion at the third node comprises installing an additional server and attaching the first or second data shuttle device to the additional server as the third node. ### Claim 9 (depends on 1) The method of claim 1, wherein the third node comprises a server and wherein restoring the third erasure-coded data portion at the third node comprises: communicatively coupling at least one of the first or second data shuttle devices to the server at the location of the third node; and copying the third erasure-coded data portion from the at least one of the first or second data shuttle devices to the server via the communicative coupling. ### Claim 10 (depends on 1) The method of claim 1, wherein the method further comprises providing, to a client of the distributed storage environment, access to the third erasure-coded data portions from at least one of the first or second data shuttles, wherein the access is provided while the at least one of the first or second data shuttles is communicatively coupled to at least one of the first, second, or third nodes. ### Claim 11 (depends on 1) The method of claim 1, wherein the first data shuttle device is included in a first set of data shuttle devices and the second data shuttle devices is included in a second set of data shuttle device, wherein generating the first restoration data on the first data shuttle device comprises copying the first erasure-coded data portion to the first set of data shuttle devices and generating the second restoration data on the second data shuttle device comprises copying the second erasure-coded data portion to the second set of data shuttle devices, further comprising: generating, on the first set of data shuttle devices subsequent to the first set of data shuttle devices being transported to the second node, the third erasure-coded data portion by applying the erasure-coding process to (i) the first erasure-coded data portion copied on the first set of data shuttle devices and (ii) a fourth erasure-coded data portion stored at the second node, wherein the third erasure-coded data portion at the third node is restored at the third node from the first set of data shuttle devices subsequent to physically transporting the first set of data shuttle devices to the location of the third node; generating, the second set of data shuttle devices subsequent to the second set of data shuttle devices being transported to the first node, a fifth erasure-coded data portion by applying the erasure-coding process to (i) the second erasure-coded data portion copied on the second set of data shuttle devices and (ii) a sixth erasure-coded data portion stored at the second node; and restoring the fifth erasure-coded data portion at the third node subsequent to physically transporting the second set of data shuttle devices to the location of the third node. ### Claim 12 A non-transitory machine-readable medium having stored thereon instructions for performing a method comprising machine-executable code that, when executed by at least one machine, causes the at least machine to: detect that a first data shuttle device is communicatively coupled to a first node of a distributed storage environment comprising the first node and a second node; generate, from a first erasure-coded data portion stored on the first node, first restoration data on the first data shuttle device responsive to detecting that the first data shuttle device is communicatively coupled to the first node; detect that the first data shuttle device or a second data shuttle device is communicatively coupled to the second node; generate, from a second erasure-coded data portion stored on the second node, second restoration data on the first or second data shuttle device communicatively coupled to the second node, wherein the second restoration data is generated responsive to detecting that the first or second data shuttle device is communicatively coupled to the second node; and restore a third erasure-coded data portion at a third node of the distributed storage environment subsequent to at least one of the first or second data shuttle devices being transported from at least one of the first or second nodes to a location of the third node, wherein the third erasure-coded data portion is generated via an erasure-coding process from at least one of the first restoration data stored on the first or second data shuttle device or the second restoration data stored on the first or second data shuttle device. ### Claim 13 (depends on 12) The non-transitory machine-readable medium of claim 12, wherein generating and storing the first restoration data comprises copying the first erasure-coded data portion to the first data shuttle device; wherein the first data shuttle device is detected as being communicatively coupled to the second node subsequent to the first restoration data being generated and the first data shuttle device being decoupled from the first node; wherein the second restoration data comprises the third erasure-coded data portion and is generated on the first data shuttle device from the first erasure-coded data portion stored on the first data shuttle device and the second erasure-coded data portion stored on the second node; wherein restoring the third erasure-coded data portion at the third node comprises copying the third erasure-coded data portion from the first data shuttle device to the third node or installing the first data shuttle device as the third node after physically transporting the first data shuttle device from the second node to the location of the third node. ### Claim 14 (depends on 13) The non-transitory machine-readable medium of claim 13, wherein generating and storing the second restoration data comprises: copying the first erasure-coded data portion from the first data shuttle device to a memory of the second node; generating the third erasure-coded data portion in the memory of the second node from the first erasure-coded data portion copied from the first data shuttle device and the second erasure-coded data portion stored at the second node; and replacing the first erasure-coded data portion stored on the first data shuttle device with the third erasure-coded data portion generated in the memory of the second node. ### Claim 15 (depends on 12) The non-transitory machine-readable medium of claim 12, wherein the machine-executable code, when executed by the at least one machine, causes the at least one machine to perform, prior to generating the first restoration data and the second restoration data, operations comprising: determining that the third node has failed; identifying the first node and the second node as respectively storing the first erasure-coded data portion and the second erasure-coded data portion; determining that the third node is to be restored using the at least one of the first data shuttle device or the second data shuttle device; and configuring the identified first and second nodes and the at least one of the first data shuttle device or the second data shuttle device for restoring the third erasure-coded data portion using the first erasure-coded data portion and the second erasure-coded data portion. ### Claim 16 A computing system comprising: a memory containing machine readable medium comprising machine-executable code having stored thereon instructions for performing a method; at least one processor coupled to the memory, the processor configured to execute the machine-executable code to cause the at least one processor to: detect that a first data shuttle device is communicatively coupled to a first node of a distributed storage environment comprising the first node and a second node, generate, from a first erasure-coded data portion stored on the first node, first restoration data on the first data shuttle device responsive to detecting that the first data shuttle device is communicatively coupled to the first node, detect that the first data shuttle device or a second data shuttle device is communicatively coupled to the second node, generate, from a second erasure-coded data portion stored on the second node, second restoration data on the first or second data shuttle device communicatively coupled to the second node, wherein the second restoration data is generated responsive to detecting that the first or second data shuttle device is communicatively coupled to the second node, and restore a third erasure-coded data portion at a third node of the distributed storage environment at a location of the third node, wherein the third erasure-coded data portion is generated via an erasure-coding process from at least one of the first restoration data stored on the first or second data shuttle device or the second restoration data stored on the first or second data shuttle device. ### Claim 17 (depends on 16) The computing system of claim 16, wherein the at least one processor is configured to generate and store the first restoration data by copying the first erasure-coded data portion to the first data shuttle device; wherein the at least one processor is configured to detect the first data shuttle device as being communicatively coupled to the second node subsequent to the first restoration data being generated and the first data shuttle device being decoupled from the first node; wherein the second restoration data comprises the third erasure-coded data portion and is generated on the first data shuttle device from the first erasure-coded data portion stored on the first data shuttle device and the second erasure-coded data portion stored on the second node; wherein the at least one processor is configured to restore the third erasure-coded data portion at the third node by copying the third erasure-coded data portion from the first data shuttle device to the third node or installing the first data shuttle device as the third node after physically transporting the first data shuttle device from the second node to the location of the third node. ### Claim 18 (depends on 17) The computing system of claim 17, wherein the at least one processor is configured to generate and store the second restoration data by performing operations comprising: copying the first erasure-coded data portion from the first data shuttle device to a memory of the second node; generating the third erasure-coded data portion in the memory of the second node from the first erasure-coded data portion copied from the first data shuttle device and the second erasure-coded data portion stored at the second node; and replacing the first erasure-coded data portion stored on the first data shuttle device with the third erasure-coded data portion generated in the memory of the second node. ### Claim 19 (depends on 16) The computing system of claim 16, wherein the at least one processor is configured to perform, prior to generating the first restoration data and the second restoration data, operations comprising: determining that the third node has failed; identifying the first node and the second node as respectively storing the first erasure-coded data portion and the second erasure-coded data portion; determining that the third node is to be restored using the at least one of the first data shuttle device or the second data shuttle device; and configuring the identified first and second nodes and the at least one of the first data shuttle device or the second data shuttle device for restoring the third erasure-coded data portion using the first erasure-coded data portion and the second erasure-coded data portion. ### Claim 20 (depends on 16) The computing system of claim 16, wherein the first data shuttle device is included in a first set of data shuttle devices and the second data shuttle devices is included in a second set of data shuttle device, wherein the at least one processor is configured to generate the first restoration data on the first data shuttle device by copying the first erasure-coded data portion to the first set of data shuttle devices and generating the second restoration data on the second data shuttle device comprises copying the second erasure-coded data portion to the second set of data shuttle devices, wherein the at least one processor is further configured to: generate, on the first set of data shuttle devices subsequent to the first set of data shuttle devices being transported to the second node, the third erasure-coded data portion by applying the erasure-coding process to (i) the first erasure-coded data portion copied on the first set of data shuttle devices and (ii) a fourth erasure-coded data portion stored at the second node, wherein the third erasure-coded data portion at the third node is restored at the third node from the first set of data shuttle devices subsequent to the first set of data shuttle devices being physically transported to the location of the third node; generate, on the second set of data shuttle devices subsequent to the second set of data shuttle devices being transported to the first node, a fifth erasure-coded data portion by applying the erasure-coding process to (i) the second erasure-coded data portion copied on the second set of data shuttle devices and (ii) a sixth erasure-coded data portion stored at the second node; and restore the fifth erasure-coded data portion at the third node subsequent to the second set of data shuttle devices being physically transported to the location of the third node.