# US 12,216,548 B2 · Storage resource resynchronization using pending IO requests > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,216,548 B2 - Document ID: US12216548B2 - Status: Granted - Kind: B2 - Application number: 18/164,382 - Filed: 2023-02-03 - Published: 2024-08-08 - Granted: 2025-02-04 - Earliest claimed: 2023-02-03 - Assignees: Dell Products L.P. - Applicants: Dell Products L.P. - Inventors: Vasudevan Subramanian; Nagapraveen Veeravenkata Seela; Michael C. Brundage; Alan L. Taylor - Web page: https://patentgrep.com/patent/US12216548B2 ## CPC - G06F 3/061: {Improving I/O performance} - G06F 11/1456: {Hardware arrangements for backup} - G06F 3/0619: {in relation to data integrity, e.g. data losses, bit errors} - G06F 3/0644: {Management of space entities, e.g. partitions, extents, pools} - G06F 3/065: {Replication mechanisms} - G06F 3/067: {Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]} - G06F 3/0683: {Plurality of storage devices} - G06F 11/2082: {Data synchronisation} - G06F 11/2094: {Redundant storage or storage space (G06F11/2056 takes precedence)} - G06F 2201/84: Using snapshots, i.e. a logical point-in-time copy of the data - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Techniques are disclosed for storage resource resynchronization using pending IO requests. For example, a method includes, upon detecting a failed network: quiescing IO activity in progress; identifying storage resources and associated pending IO requests; for each storage resource: creating a first snapshot of the storage resource; for each pending IO request, marking blocks specified by the pending IO request as modified; unquiescing the quiesced IO activity; upon detecting the network has been restored: creating a second snapshot of each storage resource; remote mirroring, to the destination, a copy of differences of each source storage resource to the destination storage resource, wherein the differences are identified by comparing the first and the second snapshot; transmitting a request for the destination to create a third snapshot comprising a snapshot of each destination storage resource; and triggering the destination to process the third snapshot to identify storage resources based on the differences. ## Claims ### Claim 1 A system comprising: at least one processing device including a processor coupled to a memory; the at least one processing device being configured to implement the following steps: in response to detecting, at a source storage system including a source storage array, a failed network connection between the source storage system and a destination storage system including a destination storage array: quiescing, at the source storage system, input/output (IO) activity in progress; identifying, at the source storage system, impacted storage resources and pending IO requests associated with each impacted storage resource; for each impacted storage resource: creating, at the source storage system, a first snapshot comprising a snapshot of the impacted storage resource resident on the source storage array; for each pending IO request associated with the impacted storage resource, marking blocks specified by the pending IO request as modified; unquiescing, at the source storage system, the quiesced IO activity; in response to detecting, at the source storage system, that the network connection has been restored between the source storage system and the destination storage system: creating, at the source storage system, a second snapshot comprising a snapshot of each impacted storage resource resident on the source storage array; performing remote mirroring, from the source storage system to the destination storage system, to store a copy of differences in data of each impacted storage resource resident on the source storage array to the impacted storage resource resident on the destination storage array, wherein the differences in data are identified based on comparing the first and the second snapshot for each impacted storage resource; transmitting, from the source storage system, a snapshot request for the destination storage system to create a third snapshot comprising a snapshot of each impacted storage resource resident on the destination storage array; and triggering, from the source storage system, the destination storage system to process the third snapshot to identify impacted storage resources for review from the destination storage array based on the differences in data from the source storage system. ### Claim 2 (depends on 1) The system of claim 1, wherein marking blocks as modified further comprises: issuing a data-less write request, at the source storage system, for a logical block address and a length specified by the pending IO request, wherein the data-less write request is configured to mark the blocks as modified in a mapper layer without writing any content to the impacted storage resource resident on the source storage array. ### Claim 3 (depends on 1) The system of claim 1, the at least one processing device being further configured to implement the following steps: deleting, at the source storage system, the first and second snapshot upon receiving, from the destination storage system, an acknowledgment that a resynchronization of the pending IO requests is complete. ### Claim 4 (depends on 1) The system of claim 1, wherein quiescing IO activity further comprises temporarily pausing incoming requests for storage resources that have metro replication or sync replication enabled to the destination storage system, and wherein unquiescing the quiesced IO activity further comprises resuming processing of the incoming requests. ### Claim 5 (depends on 1) The system of claim 1, wherein quiescing IO activity further comprises freezing an acknowledgment from the source storage system to hosts specified by pending IO requests that have incomplete remote requests for storage resources with metro replication or sync replication enabled to the destination storage system, and wherein unquiescing the quiesced IO activity further comprises sending the acknowledgment from the source storage system to the specified hosts. ### Claim 6 (depends on 1) The system of claim 1, wherein the impacted storage resource comprises a volume, a file system, a logical unit (LUN), or a virtual volume (vVol). ### Claim 7 (depends on 1) The system of claim 1, wherein the impacted storage resource has metro replication or sync replication enabled. ### Claim 8 (depends on 7) The system of claim 7, wherein, if the impacted storage resource has metro replication enabled, then the source storage array is designated as Preferred and the destination storage array is designated as Non-preferred. ### Claim 9 (depends on 7) The system of claim 7, wherein, if the impacted storage resource has sync replication enabled, then the source storage array is designated as Active. ### Claim 10 A method comprising: in response to detecting, at a source storage system including a source storage array, a failed network connection between the source storage system and a destination storage system including a destination storage array: quiescing, at the source storage system, input/output (IO) activity in progress; identifying, at the source storage system, impacted storage resources and pending IO requests associated with each impacted storage resource; for each impacted storage resource: creating, at the source storage system, a first snapshot comprising a snapshot of the impacted storage resource resident on the source storage array; for each pending IO request associated with the impacted storage resource, marking blocks specified by the pending IO request as modified; unquiescing, at the source storage system, the quiesced IO activity; in response to detecting, at the source storage system, that the network connection has been restored between the source storage system and the destination storage system: creating, at the source storage system, a second snapshot comprising a snapshot of each impacted storage resource resident on the source storage array; performing remote mirroring, from the source storage system to the destination storage system, to store a copy of differences in data of each impacted storage resource resident on the source storage array to the impacted storage resource resident on the destination storage array, wherein the differences in data are identified based on comparing the first and the second snapshot for each impacted storage resource; transmitting, from the source storage system, a snapshot request for the destination storage system to create a third snapshot comprising a snapshot of each impacted storage resource resident on the destination storage array; and triggering, from the source storage system, the destination storage system to process the third snapshot to identify impacted storage resources for review from the destination storage array based on the differences in data from the source storage system. ### Claim 11 (depends on 10) The method of claim 10, wherein marking blocks as modified further comprises: issuing a data-less write request, at the source storage system, for a logical block address and a length specified by the pending IO request, wherein the data-less write request is configured to mark the blocks as modified in a mapper layer without writing any content to the impacted storage resource resident on the source storage array. ### Claim 12 (depends on 10) The method of claim 10, further comprising: deleting, at the source storage system, the first and second snapshot upon receiving, from the destination storage system, an acknowledgment that a resynchronization of the pending IO requests is complete. ### Claim 13 (depends on 10) The method of claim 10, wherein quiescing IO activity further comprises temporarily pausing incoming requests for storage resources that have metro replication or sync replication enabled to the destination storage system, and wherein unquiescing the quiesced IO activity further comprises resuming processing of the incoming requests. ### Claim 14 (depends on 10) The method of claim 10, wherein quiescing IO activity further comprises freezing an acknowledgment from the source storage system to hosts specified by pending IO requests that have incomplete remote requests for storage resources with metro replication or sync replication enabled to the destination storage system, and wherein unquiescing the quiesced IO activity further comprises sending the acknowledgment from the source storage system to the specified hosts. ### Claim 15 (depends on 10) The method of claim 10, wherein the impacted storage resource comprises a volume, a file system, a logical unit (LUN), or a virtual volume (vVol). ### Claim 16 (depends on 10) The method of claim 10, wherein the impacted storage resource has metro replication or sync replication enabled. ### Claim 17 (depends on 16) The method of claim 16, wherein, if the impacted storage resource has metro replication enabled, then the source storage array is designated as Preferred and the destination storage array is designated as Non-preferred. ### Claim 18 (depends on 16) The method of claim 16, wherein, if the impacted storage resource has sync replication enabled, then the source storage array is designated as Active. ### Claim 19 A non-transitory processor-readable storage medium having stored thereon program code of one or more software programs, wherein the program code when executed by at least one processing device causes the at least one processing device to perform the following steps: in response to detecting, at a source storage system including a source storage array, a failed network connection between the source storage system and a destination storage system including a destination storage array: quiescing, at the source storage system, input/output (IO) activity in progress; identifying, at the source storage system, impacted storage resources and pending IO requests associated with each impacted storage resource; for each impacted storage resource: creating, at the source storage system, a first snapshot comprising a snapshot of the impacted storage resource resident on the source storage array; for each pending IO request associated with the impacted storage resource, marking blocks specified by the pending IO request as modified; unquiescing, at the source storage system, the quiesced IO activity; in response to detecting, at the source storage system, that the network connection has been restored between the source storage system and the destination storage system: creating, at the source storage system, a second snapshot comprising a snapshot of each impacted storage resource resident on the source storage array; performing remote mirroring, from the source storage system to the destination storage system, to store a copy of differences in data of each impacted storage resource resident on the source storage array to the impacted storage resource resident on the destination storage array, wherein the differences in data are identified based on comparing the first and the second snapshot for each impacted storage resource; transmitting, from the source storage system, a snapshot request for the destination storage system to create a third snapshot comprising a snapshot of each impacted storage resource resident on the destination storage array; and triggering, from the source storage system, the destination storage system to process the third snapshot to identify impacted storage resources for review from the destination storage array based on the differences in data from the source storage system. ### Claim 20 (depends on 19) The non-transitory processor-readable storage medium of claim 19, wherein marking blocks as modified further comprises: issuing a data-less write request, at the source storage system, for a logical block address and a length specified by the pending IO request, wherein the data-less write request is configured to mark the blocks as modified in a mapper layer without writing any content to the impacted storage resource resident on the source storage array.