# US 9,436,564 B1 · Creating distributed storage during partitions > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,436,564 B1 - Document ID: US9436564B1 - Status: Granted - Kind: B1 - Application number: 14/230,545 - Filed: 2014-03-31 - Granted: 2016-09-06 - Earliest claimed: 2014-03-31 - Assignees: EMC Corporation - Applicants: EMC Corporation - Inventors: Roel van der Goot - Web page: https://patentgrep.com/patent/US9436564B1 ## CPC - G06F 11/1076: {Parity data used in redundant arrays of independent storages, e.g. in RAID systems} - G06F 11/2094: {Redundant storage or storage space (G06F11/2056 takes precedence)} - G06F 3/0611: {in relation to response time} - G06F 3/0647: {Migration mechanisms} - G06F 3/067: {Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]} - G06F 11/00: Error detection; Error correction; Monitoring (error detection, correction or monitoring in information storage based on relative movement between record carrier and transducer G11B20/18; monitoring, i.e. supervising the progress of recording or reproducing G11B27/36; in static stores G11C29/00) - G06F 11/1471: involving logging of persistent data for recovery - G06F 11/2089: {Redundant storage control functionality} - G06F 12/0813: with a network or matrix configuration - G06F 12/0815: Cache consistency protocols - G06F 2201/805: Real-time - G06F 2201/82: Solving problems relating to consistency - G06F 2212/154: Networked environment - G06F 2212/60: Details of cache memory - G06F 2212/621: Coherency control relating to peripheral accessing, e.g. from DMA or I/O device - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A system and method are provided for processing to create distributed volume in a distributed storage system during a failure that has partitioned the distributed volume (e.g. an array failure, a site failure and/or an inter-site network failure). In an embodiment, the system described herein may provide for continuing distributed storage processing in response to I/O requests from a source by creating the local parts of the distributed storage during the failure, and, when the remote site or inter-site network return to availability, the remaining part of the distributed volume is automatically created. The system may include an automatic rebuild to make sure that all parts of the distributed volume are consistent again. The processing may be transparent to the source of the I/O requests. ## Claims ### Claim 1 A method of maintaining distributed storage processing of a distributed volume, the method being performed by at least one processor and comprising: determining occurrence of a failure causing at least one array of a plurality of arrays of a distributed storage system to become unavailable; identifying at least one available array of the plurality of arrays; creating a local part of the distributed volume and indicating the distributed volume as being out of compliance with distributed storage requirements; exporting the local part of the distributed volume to the at least one available array; in response to at least one I/O request received at the distributed storage system for the distributed volume during the failure, continuing the distributed storage processing at the local part of the distributed volume and tracking changes to the local part of the distributed volume; determining that the at least one unavailable array of the plurality of arrays is again available to become a reappeared array; after the at least one unavailable array is determined as being again available, creating a remote part of the distributed volume at the reappeared array; performing consistency processing to ensure that the distributed volume is in compliance with the distributed storage requirements; after creating the local part, generating a report indicating the distributed volume as being out of compliance with distributed storage requirements; and after performing the consistency processing, generating a report indicating repair of the distributed storage processing. ### Claim 2 (depends on 1) The method according to claim 1, wherein the distributed storage system includes a redundant array of independent disks (RAID) system. ### Claim 3 (depends on 1) The method according to claim 1, wherein performance of the method is transparent to a source of the at least one I/O request. ### Claim 4 (depends on 1) The method according to claim 1, wherein the consistency processing includes performing an automatic rebuild of the distributed volume to ensure the remote part of the distributed volume is made consistent with the local part. ### Claim 5 (depends on 1) The method according to claim 1, wherein a designation of the distributed volume as being out of compliance with the distributed storage requirements that occurs after creation of the local part during the failure is the same as a designation of a particular distributed volume being out of compliance resulting from the failure occurring after creation of the particular distributed volume. ### Claim 6 A non-transitory computer-readable medium storing software for maintaining distributed storage processing of a distributed volume, the software comprising: executable code that determines occurrence of a failure causing at least one array of a plurality of arrays of a distributed storage system to become unavailable; executable code that identifies at least one available array of the plurality of arrays; executable code that creates a local part of the distributed volume and indicates the distributed volume as being out of compliance with distributed storage requirements; executable code that exports the local part of the distributed volume to the at least one available array; executable code that, in response to at least one I/O request received at the distributed storage system for the distributed volume during the failure, continues the distributed storage processing at the local part of the distributed volume and tracking changes to the local part of the distributed volume; executable code that determines that the at least one unavailable array of the plurality of arrays is again available to become a reappeared array; executable code that, after the at least one unavailable array is determined as being again available, creates a remote part of the distributed volume at the reappeared array; and executable code that performs consistency processing to ensure that the distributed volume is in compliance with the distributed storage requirements; after performing the consistency processing, generating a report indicating repair of the distributed storage processing; and executable code that, after performing the consistency processing, generates a report indicating the failure and the steps performed to maintain the distributed storage processing. ### Claim 7 (depends on 6) The non-transitory computer-readable medium according to claim 6, wherein the distributed storage system includes a redundant array of independent disks (RAID) system. ### Claim 8 (depends on 6) The non-transitory computer-readable medium according to claim 6, wherein performance of the software is transparent to a source of the at least one I/O request. ### Claim 9 (depends on 6) The non-transitory computer-readable medium according to claim 6, wherein the executable code that performs consistency processing includes executable code that performs an automatic rebuild of the distributed volume to ensure the remote part of the distributed volume is made consistent with the local part. ### Claim 10 (depends on 6) The non-transitory computer-readable medium according to claim 6, wherein a designation of the distributed volume as being out of compliance with the distributed storage requirements that occurs after creation of the local part during the failure is the same as a designation of a particular distributed volume being out of compliance resulting from the failure occurring after creation of the particular distributed volume. ### Claim 11 A distributed storage processing system, comprising: a plurality of arrays that store data; at least one controller that manages storage of data on the plurality of arrays; and a non-transitory computer-readable medium storing software, executable by the at least one controller, for maintaining distributed storage processing of a distributed volume during a failure in connection with a plurality of arrays of a distributed storage system, the software comprising: executable code that determines occurrence of a failure causing at least one array of a plurality of arrays of a distributed storage system to become unavailable; executable code that identifies at least one available array of the plurality of arrays; executable code that creates a local part of the distributed volume and indicates the distributed volume as being out of compliance with distributed storage requirements; executable code that exports the local part of the distributed volume to the at least one available array; executable code that, in response to at least one I/O request received at the distributed storage system for the distributed volume during the failure, continues the distributed storage processing at the local part of the distributed volume and tracking changes to the local part of the distributed volume; executable code that determines that the at least one unavailable array of the plurality of arrays is again available to become a reappeared array; executable code that, after the at least one unavailable array is determined as being again available, creates a remote part of the distributed volume at the reappeared array; executable code that performs consistency processing to ensure that the distributed volume is in compliance with the distributed storage requirements; executable code that, after creating the local part, generates a report indicating the distributed volume as being out of compliance with distributed storage requirements; and executable code that, after performing the consistency processing, generates a report indicating the failure and the steps performed to maintain the distributed storage processing. ### Claim 12 (depends on 11) The distributed storage system according to claim 11, wherein the plurality of arrays includes a redundant array of independent disks (RAID) system. ### Claim 13 (depends on 11) The distributed storage system according to claim 11, wherein performance of the software is transparent to a source of the at least one I/O request. ### Claim 14 (depends on 11) The distributed storage system according to claim 11, wherein the executable code that performs consistency processing includes executable code that performs an automatic rebuild of the distributed volume to ensure the remote part of the distributed volume is made consistent with the local part. ### Claim 15 (depends on 11) The distributed storage system according to claim 11, wherein a designation of the distributed volume as being out of compliance with the distributed storage requirements that occurs after creation of the local part during the failure is the same as a designation of a particular distributed volume being out of compliance resulting from the failure occurring after creation of the particular distributed volume.