# US 10,871,911 B2 · Reducing data amplification when replicating objects across different sites > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,871,911 B2 - Document ID: US10871911B2 - Status: Granted - Kind: B2 - Application number: 15/660,682 - Filed: 2017-07-26 - Published: 2019-01-31 - Granted: 2020-12-22 - Earliest claimed: 2017-07-26 - Assignees: VMware, Inc. - Applicants: VMware, Inc. - Inventors: Pascal Renauld; Enning Xiang; Eric Knauft - Web page: https://patentgrep.com/patent/US10871911B2 ## CPC - G06F 3/065: {Replication mechanisms} - G06F 3/0619: {in relation to data integrity, e.g. data losses, bit errors} - G06F 3/0664: {at device level, e.g. emulation of a storage device or system} - G06F 3/0688: {Non-volatile semiconductor memory arrays} - G06F 3/0689: {Disk arrays, e.g. RAID, JBOD} - G06F 9/45558: {Hypervisor-specific management and integration aspects} - G06F 11/1076: {Parity data used in redundant arrays of independent storages, e.g. in RAID systems} - G06F 11/1464: {for networked environments} - G06F 11/2058: {using more than 2 mirrored copies} - G06F 11/2094: {Redundant storage or storage space (G06F11/2056 takes precedence)} - G06F 16/27: Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor - G06F 2009/45579: {I/O management, e.g. providing access to device drivers or storage} - G06F 2009/45583: {Memory management, e.g. access or allocation} - G06F 2009/45595: {Network integration; Enabling network access in virtual machine instances} - G06F 2201/815: Virtual - G06F 2201/84: Using snapshots, i.e. a logical point-in-time copy of the data - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Component objects of a virtual disk are backed by first storage nodes, which are at a primary site, and replicated in second storage nodes, which are at a secondary site. Responsive to a write command issued to the virtual disk, a coordinating node at the primary site issues a first write operation to a first replication module running therein, and transmits a second write operation to a second replication module running in a coordinating node at the secondary site. The first replication module, responsive to the first write operation, instructs the first storage nodes backing first and second component objects of the virtual disk to write the data. The second replication module, responsive to the second write operation, instructs the second storage nodes backing third and fourth component objects, which are replicas of the first and second component objects, respectively, to write the data. ## Claims ### Claim 1 A method of replicating component objects of a virtual disk, that are backed by first storage nodes, which are at a primary site, in second storage nodes, which are at a secondary site, said method comprising: at a coordinating node at the primary site, responsive to receiving a write command issued to the virtual disk, (i) issuing a first write operation and a replica of the first write operation, (ii) transmitting a second write operation, identical to the first write operation, to a coordinating node at the secondary site responsible for replicating data received from the coordinating node at the primary site on the second storage nodes, and (iii) instructing, responsive to the first write operation and the replica of the first write operation, the first storage nodes that are backing first and second component objects of the virtual disk to write data on at least two corresponding local storage devices, wherein the coordinating node at the secondary site, responsive to receiving the second write operation, (i) issues a replica of the second write operation and (ii) instructs the second storage nodes that are backing third and fourth component objects to write data on at least two corresponding local storage devices. ### Claim 2 (depends on 1) The method of claim 1, wherein the data written on the at least two corresponding local storage devices of the first storage nodes are identical to the data written on the at least two corresponding local storage devices of the second storage nodes. ### Claim 3 (depends on 2) The method of claim 2, wherein the data written on each of the at least two corresponding local storage devices of the first storage nodes are identical, and the data written on each of the at least two corresponding local storage devices of the second storage nodes are identical. ### Claim 4 (depends on 2) The method of claim 2, wherein the data written on the at least two corresponding local storage devices of the first storage nodes include first data, which is written in at least one local storage device of one of the first storage nodes, and second data, which is written in at least one local storage device of another one of the first storage nodes, and the data written on the at least two corresponding local storage devices of the second storage nodes include the first data and the second data, the first data being written in at least one local storage device of one of the second storage nodes and the second data being written in at least one local storage device of another one of the second storage nodes. ### Claim 5 (depends on 4) The method of claim 4, wherein the first data is user data and the second data is parity data. ### Claim 6 (depends on 1) The method of claim 1, wherein the instructing by the coordinating nodes at the primary and the secondary sites are performed by their respective RAID 1 engines. ### Claim 7 (depends on 1) The method of claim 1, wherein the instructing by the coordinating nodes at the primary and the secondary sites are performed by their respective RAID 5 engines. ### Claim 8 (depends on 1) The method of claim 1, wherein the coordinating node at the primary site further includes a replication module that issues the first and second write operations. ### Claim 9 A non-transitory computer readable medium comprising instructions to be executed in a coordinating node at a primary site to carry out a method of replicating component objects of a virtual disk, that are backed by first storage nodes, which are at the primary site, in second storage nodes, which are at a secondary site, said method comprising: responsive to receiving a write command issued to the virtual disk, (i) issuing a first write operation and a replica of the first write operation, (ii) transmitting a second write operation, identical to the first write operation, to a coordinating node at the secondary site responsible for replicating data received from the coordinating node at the primary site on the second storage nodes, and (iii) instructing, responsive to the first write operation and the replica of the first write operation, the first storage nodes that are backing first and second component objects of the virtual disk to write data on at least two corresponding local storage devices, wherein the coordinating node at the secondary site, responsive to receiving the second write operation, (i) issues a replica of the second write operation and (ii) instructs the second storage nodes that are backing third and fourth component objects to write data on at least two corresponding local storage devices. ### Claim 10 (depends on 9) The non-transitory computer readable medium of claim 9, wherein the data written on the at least two corresponding local storage devices of the first storage nodes are identical to the data written on the at least two corresponding local storage devices of the second storage nodes. ### Claim 11 (depends on 10) The non-transitory computer readable medium of claim 10, wherein the data written on each of the at least two corresponding local storage devices of the first storage nodes are identical, and the data written on each of the at least two corresponding local storage devices of the second storage nodes are identical. ### Claim 12 (depends on 10) The non-transitory computer readable medium of claim 10, wherein the data written on the at least two corresponding local storage devices of the first storage nodes include first data, which is written in at least one local storage device of one of the first storage nodes, and second data, which is written in at least one local storage device of another one of the first storage nodes, and the data written on the at least two corresponding local storage devices of the second storage nodes include the first data and the second data, the first data being written in at least one local storage device of one of the second storage nodes and the second data being written in at least one local storage device of another one of the second storage nodes. ### Claim 13 (depends on 12) The non-transitory computer readable medium of claim 12, wherein the first data is user data and the second data is parity data. ### Claim 14 (depends on 9) The non-transitory computer readable medium of claim 9, wherein the instructing by the coordinating nodes at the primary and the secondary sites are performed by their respective RAID 1 engines. ### Claim 15 (depends on 9) The non-transitory computer readable medium of claim 9, wherein the instructing by the coordinating nodes at the primary and the secondary sites are performed by their respective RAID 5 engines. ### Claim 16 (depends on 9) The non-transitory computer readable medium of claim 9, wherein the coordinating node at the primary site further includes a replication module that issues the first and second write operations. ### Claim 17 A distributed computer system for replicating first and second component objects of a virtual disk, comprising: a first coordinating node; first storage nodes that back the first and second component objects, the first storage nodes communicating with the first coordinating node through a first local area network; a second coordinating node communicating with the first coordinating node through a wide area network; and second storage nodes that back third and fourth component objects, which are replicas of the first and second component objects, respectively, the second storage nodes communicating with the second coordinating node through a second local area network, wherein responsive to receiving a write command issued to the virtual disk, the first coordinating node (i) issues a first write operation and a replica of the first write operation, (ii) transmits a second write operation, identical to the first write operation, to the second coordinating node responsible for replicating data received from the first coordinating node on the second storage nodes, and (iii) instructs, responsive to the first write operation and the replica of the first write operation, the first storage nodes that are backing the first and second component objects to write data on at least two corresponding local storage devices, wherein the second coordinating node, responsive to receiving the second write operation, (i) issues a replica of the second write operation and (ii) instructs the second storage nodes that are backing the third and fourth component objects to write data on at least two corresponding local storage devices of the second storage nodes. ### Claim 18 (depends on 17) The distributed computer system of claim 17, wherein the instructing by the coordinating nodes at the primary and the secondary sites are performed by their respective RAID 1 engines. ### Claim 19 (depends on 17) The distributed computer system of claim 17, wherein the instructing by the coordinating nodes at the primary and the secondary sites are performed by their respective RAID 5 engines. ### Claim 20 (depends on 17) The distributed computer system of claim 17, wherein the first coordinating node further includes a RAID 1 engine that issues the first and second write operations.