# US 2017/0286436 A1 · System and method for fast parallel data processing in distributed storage systems > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2017/0286436 A1 - Document ID: US20170286436A1 - Status: Published application - Kind: A1 - Application number: 15/086,891 - Filed: 2016-03-31 - Published: 2017-10-05 - Earliest claimed: 2016-03-31 - Assignees: Acronis International GmbH - Applicants: Acronis International GmbH - Inventors: Andrei Neporada; Vladimir Simonov; Stanislav Protasov; Mark Smulevich; Serguei Beloussov - Web page: https://patentgrep.com/patent/US20170286436A1 ## CPC - G06F 16/113: {Details of archiving (lifecycle management in storage systems G06F3/0649; point-in-time backing up or restoration of persistent data G06F11/1446)} - G06F 3/061: {Improving I/O performance} - G06F 3/064: {Management of blocks} - G06F 3/067: {Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]} - G06F 16/1727: {Details of free space management performed by the file system (saving storage space on storage systems G06F3/0608; management of blocks in storage devices G06F3/064)} - G06F 16/182: Distributed file systems - G06F 16/278: {Data partitioning, e.g. horizontal or vertical partitioning} - H03M 13/154: {Error and erasure correction, e.g. by using the error and erasure locator or Forney polynomial} - H03M 13/353: {Adaptation to the channel} - H03M 13/373: {with erasure correction and erasure determination, e.g. for packet loss recovery or setting of erasures for the decoding of Reed-Solomon codes} - H03M 13/3761: {using code combining, i.e. using combining of codeword portions which may have been transmitted separately, e.g. Digital Fountain codes, Raptor codes or Luby Transform [LT] codes} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H03M: CODING; DECODING; CODE CONVERSION IN GENERAL (using fluidic means F15C4/00; optical analogue/digital converters G02F7/00; coding, decoding or code conversion, specially adapted for particular applications, see the relevant subclasses, e.g. G01D, G01R, G06F, G06T, G09G, G10L, G11B, G11C, H04B, H04L, H04M, H04N; ciphering or deciphering for cryptography or other purposes involving the need for secrecy G09C) ## Abstract A system and method is disclosed for fast parallel data processing in a distributed storage system. An example method includes receiving at least one digital object; determining, by a processor of a computing device, whether the at least one digital object has a fixed data block structure; in response to determining that the at least one digital object has a fixed data block structure, determining by the processor a size of each fixed data block of the at least one digital object; determining a number of archive chunks for storing the at least one digital object on a selected plurality of storage servers based at least upon the size of each fixed data block; and partitioning the at least one digital object into the number of archive chunks for storing on the selected plurality of storage servers using erasure coding. ## Claims ### Claim 1 A method for fast parallel data processing in a distributed storage system, the method comprising: receiving at least one digital object; determining, by a processor of a computing device, whether the at least one digital object has a fixed data block structure; in response to determining that the at least one digital object has a fixed data block structure, determining by the processor a size of each fixed data block of the at least one digital object; determining a number of archive chunks for storing the at least one digital object on a selected plurality of storage servers based at least upon the size of each fixed data block; and partitioning the at least one digital object into the number of archive chunks for storing on the selected plurality of storage servers using erasure coding. ### Claim 2 (depends on 1) The method of claim 1, further comprising adjusting parameters of the erasure coding for the selected plurality of storage servers based on a criticality of data stored thereon. ### Claim 3 (depends on 1) The method of claim 1, further comprising: in response to detecting that the at least one digital object comprises two or more files or records having different sizes, determining that the at least one digital object has an irregular structure. ### Claim 4 (depends on 1) The method of claim 1, wherein the number of archive chunks is a multiple of the size of each fixed data block. ### Claim 5 (depends on 1) The method of claim 1, further comprising: in response to determining that the at least one digital object has the fixed data block structure, analyzing an application generating the at least one digital object to determine the size of each fixed data block; and partitioning the at least one digital object into a plurality of parts based on the size of each fixed data block while keeping the at least one digital object logically intact. ### Claim 6 (depends on 1) The method of claim 1, further comprising: in response to receiving a data processing request relating to the at least one digital object, retrieving the number of archive chunks from the selected plurality of storage servers for forming the at least one digital object. ### Claim 7 (depends on 2) The method of claim 2, wherein adjusting the parameters of the erasure coding comprises adjusting at least the number of archive chunks for storing the at least one digital object on the selected plurality of storage servers to achieve a selected overhead. ### Claim 8 A distributed storage system for fast parallel data processing, the system comprising: an electronic memory; and a hardware processor couple to the electronic memory and configured to: receive at least one digital object; determine whether the at least one digital object has a fixed data block structure; in response to determining that the at least one digital object has a fixed data block structure, determine a size of each fixed data block of the at least one digital object; determine a number of archive chunks for storing the at least one digital object on a selected plurality of storage servers based at least upon the size of each fixed data block; and partition the at least one digital object into the number of archive chunks for storing on the selected plurality of storage servers using erasure coding. ### Claim 9 (depends on 8) The system of claim 8, wherein the processor is further configured to adjust parameters of the erasure coding for the selected plurality of storage servers based on a criticality of data stored thereon. ### Claim 10 (depends on 8) The system of claim 8, wherein the processor is further configured to, in response to detecting that the at least one digital object comprises two or more files or records having different sizes, determine that the at least one digital object has an irregular structure. ### Claim 11 (depends on 8) The system of claim 8, wherein the number of archive chunks is a multiple of the size of each fixed data block. ### Claim 12 (depends on 8) The system of claim 8, wherein the processor is further configured to: in response to determining that the at least one digital object has the fixed data block structure, analyze an application generating the at least one digital object to determine the size of each fixed data block; and partition the at least one digital object into a plurality of parts based on the size of each fixed data block while keeping the at least one digital object logically intact. ### Claim 13 (depends on 8) The system of claim 8, wherein the processor is further configured to, in response to receiving a data processing request relating to the at least one digital object, retrieve the number of archive chunks from the selected plurality of storage servers for forming the at least one digital object. ### Claim 14 (depends on 8) The system of claim 8, wherein the processor is further configured to restore a plurality of sectors of the computing device based on a priority determined based at least upon a usage frequency of each sector. ### Claim 15 (depends on 9) The system of claim 9, wherein, to adjust the parameters of the erasure coding, the processor is further configured to adjust at least the number of archive chunks for storing the at least one digital object on the selected plurality of storage servers to achieve a selected overhead ### Claim 16 A non-transitory computer readable medium storing computer executable instructions for fast parallel data processing in a distributed storage system, the instructions being configured for: receiving at least one digital object; determining, by a processor of a computing device, whether the at least one digital object has a fixed data block structure; in response to determining that the at least one digital object has a fixed data block structure, determining by the processor a size of each fixed data block of the at least one digital object; determining a number of archive chunks for storing the at least one digital object on a selected plurality of storage servers based at least upon the size of each fixed data block; and partitioning the at least one digital object into the number of archive chunks for storing on the selected plurality of storage servers using erasure coding. ### Claim 17 (depends on 16) The non-transitory computer readable medium of claim 16, further comprising instructions for: adjusting parameters of the erasure coding for the selected plurality of storage servers based on a criticality of data stored thereon; and adjusting at least the number of archive chunks for storing the at least one digital object on the selected plurality of storage servers to achieve a selected overhead. ### Claim 18 (depends on 16) The non-transitory computer readable medium of claim 16, further comprising instructions for, in response to detecting that the at least one digital object comprises two or more files or records having different sizes, determining that the at least one digital object has an irregular structure. ### Claim 19 (depends on 16) The non-transitory computer readable medium of claim 16, wherein the number of archive chunks is a multiple of the size of each fixed data block. ### Claim 20 (depends on 16) The non-transitory computer readable medium of claim 16, further comprising instructions for, in response to receiving a data processing request relating to the at least one digital object, retrieving the number of archive chunks from the selected plurality of storage servers for forming the at least one digital object.