US 10,761,765 B2Granted

Distributed object replication architecture

AssigneeEMC IP Holding Company LLC

InventorsAbhinav Duggal, Atul Avinash Karmarkar, Philip Shilane, Kevin Xu

Life of this patent

Estimating the term…

Loading patent…

Abstract

A source site includes a controller, a set of source worker nodes, and a message queue connected between the controller and source worker nodes. A destination site includes a set of destination worker nodes. The controller identifies differences between a first snapshot created at the source site at a first time and a second snapshot created at a second time, after the first time. Based on the differences, a set of tasks are generated. The tasks include one or more of copying an object from the source to destination or deleting an object from the destination. The controller places the tasks onto the message queue. A first source worker node retrieves the first task and coordinates with a first destination worker node to perform the first task. A second source worker nodes retrieves the second task and coordinates with a second destination worker node to perform the second task.

Claims15

Dependent claims

A method of replicating data comprising:

providing, at a source site, a controller, a plurality of source worker nodes, and a message queue, coupled between the controller and the plurality of source worker nodes;
providing, at a destination site, a plurality of destination worker nodes;
defining a replication context specifying a replication frequency, and information identifying objects at the source site to replicate;
identifying, by the controller, differences between a first snapshot created at the source site at a first time and a second snapshot created at the source site at a second time, after the first time;
based on the differences, generating a set of tasks comprising one or more of copying an object from the source site to the destination site, or deleting an object from the destination site;
placing, by the controller, the tasks onto the message queue;
retrieving, by a first source worker node, a first task from the message queue;
coordinating, by the first source worker node, with a first destination worker node to perform the first task;
retrieving, by a second source worker node, a second task from the message queue;
and
coordinating, by the second source worker node, with a second destination worker node to perform the second task, wherein the first snapshot comprises a first listing of objects stored in an object storage at the source site as of the first time, the second snapshot comprises a second listing of objects stored in the source object storage as of the second time, and the method comprises:
receiving the first and second listings in an alphabetical sort;
comparing the alphabetically sorted first and second listings;
determining from the comparison that the first listing includes an object not included in the second listing;
and
upon the determination, generating the first task, the first task comprising an instruction to delete the object from the destination site.

13 more elements

A method of replicating data comprising:

providing, at a source site, a controller, a plurality of source worker nodes, and a message queue, coupled between the controller and the plurality of source worker nodes;
providing, at a destination site, a plurality of destination worker nodes;
defining a replication context specifying a replication frequency, and information identifying objects at the source site to replicate;
identifying, by the controller, differences between a first snapshot created at the source site at a first time and a second snapshot created at the source site at a second time, after the first time;
based on the differences, generating a set of tasks comprising one or more of copying an object from the source site to the destination site, or deleting an object from the destination site;
placing, by the controller, the tasks onto the message queue;
retrieving, by a first source worker node, a first task from the message queue;
coordinating, by the first source worker node, with a first destination worker node to perform the first task;
retrieving, by a second source worker node, a second task from the message queue;
and
coordinating, by the second source worker node, with a second destination worker node to perform the second task, wherein the first snapshot comprises a first listing of objects stored in an object storage at the source site as of the first time, the second snapshot comprises a second listing of objects stored in the object storage as of the second time, and the method comprises:
receiving the first and second listings in an alphabetical sort;
comparing the alphabetically sorted first and second listings;
determining from the comparison that an object listed in both the first and second listings has a timestamp that changed between the first and second snapshots, the object listed in the first listing having an earlier timestamp, and the object listed in the second listing having a later timestamp;
and
upon the determination, generating the first task and the second task, the first task comprising an instruction to copy the object with the later timestamp to the destination site, and the second task comprising an instruction to delete the object with the earlier timestamp from the destination site.

13 more elements

A system for replicating data, the system comprising:

a processor, and memory storing one or more sequences of instructions which, when executed by the processor, cause the processor to carry out the steps of:
providing, at a source site, a controller, a plurality of source worker nodes, and a message queue, coupled between the controller and the plurality of source worker nodes;
providing, at a destination site, a plurality of destination worker nodes;
defining a replication context specifying a replication frequency, and information identifying objects at the source site to replicate;
identifying, by the controller, differences between a first snapshot created at the source site at a first time and a second snapshot created at the source site at a second time, after the first time;
based on the differences, generating a set of tasks comprising one or more of copying an object from the source site to the destination site, or deleting an object from the destination site;
placing, by the controller, the tasks onto the message queue;
retrieving, by a first source worker node, a first task from the message queue;
coordinating, by the first source worker node, with a first destination worker node to perform the first task;
retrieving, by a second source worker node, a second task from the message queue;
and
coordinating, by the second source worker node, with a second destination worker node to perform the second task, wherein the first snapshot comprises a first listing of objects stored in an object storage at the source site as of the first time, the second snapshot comprises a second listing of objects stored in the source object storage as of the second time, and the processor further carries out the steps of:
receiving the first and second listings in an alphabetical sort;
comparing the alphabetically sorted first and second listings;
determining from the comparison that the second listing includes an object not included in the first listing;
and
upon the determination, generating the first task, the first task comprising an instruction to copy the object to the destination site.

14 more elements

A system for replicating data, the system comprising:

a processor, and memory storing one or more sequences of instructions which, when executed by the processor, cause the processor to carry out the steps of:
providing, at a source site, a controller, a plurality of source worker nodes, and a message queue, coupled between the controller and the plurality of source worker nodes;
providing, at a destination site, a plurality of destination worker nodes;
defining a replication context specifying a replication frequency, and information identifying objects at the source site to replicate;
identifying, by the controller, differences between a first snapshot created at the source site at a first time and a second snapshot created at the source site at a second time, after the first time;
based on the differences, generating a set of tasks comprising one or more of copying an object from the source site to the destination site, or deleting an object from the destination site;
placing, by the controller, the tasks onto the message queue;
retrieving, by a first source worker node, a first task from the message queue;
coordinating, by the first source worker node, with a first destination worker node to perform the first task;
retrieving, by a second source worker node, a second task from the message queue;
and
coordinating, by the second source worker node, with a second destination worker node to perform the second task, wherein the first snapshot comprises a first listing of objects stored in an object storage at the source site as of the first time, the second snapshot comprises a second listing of objects stored in the source object storage as of the second time, and the processor further carries out the steps of:
receiving the first and second listings in an alphabetical sort;
comparing the alphabetically sorted first and second listings;
determining from the comparison that the first listing includes an object not included in the second listing;
and
upon the determination, generating the first task, the first task comprising an instruction to delete the object from the destination site.

14 more elements

A system for replicating data, the system comprising:

a processor, and memory storing one or more sequences of instructions which, when executed by the processor, cause the processor to carry out the steps of:
providing, at a source site, a controller, a plurality of source worker nodes, and a message queue, coupled between the controller and the plurality of source worker nodes;
providing, at a destination site, a plurality of destination worker nodes;
defining a replication context specifying a replication frequency, and information identifying objects at the source site to replicate;
identifying, by the controller, differences between a first snapshot created at the source site at a first time and a second snapshot created at the source site at a second time, after the first time;
based on the differences, generating a set of tasks comprising one or more of copying an object from the source site to the destination site, or deleting an object from the destination site;
placing, by the controller, the tasks onto the message queue;
retrieving, by a first source worker node, a first task from the message queue;
coordinating, by the first source worker node, with a first destination worker node to perform the first task;
retrieving, by a second source worker node, a second task from the message queue;
and
coordinating, by the second source worker node, with a second destination worker node to perform the second task, wherein the first snapshot comprises a first listing of objects stored in an object storage at the source site as of the first time, the second snapshot comprises a second listing of objects stored in the object storage as of the second time, and the processor further carries out the steps of:
receiving the first and second listings in an alphabetical sort;
comparing the alphabetically sorted first and second listings;
determining from the comparison that an object listed in both the first and second listings has a timestamp that changed between the first and second snapshots, the object listed in the first listing having an earlier timestamp, and the object listed in the second listing having a later timestamp;
and
upon the determination, generating the first task and the second task, the first task comprising an instruction to copy the object with the later timestamp to the destination site, and the second task comprising an instruction to delete the object with the earlier timestamp from the destination site.

14 more elements

A computer program product comprising a non-transitory computer-readable medium having a computer-readable program code embodied therein, the computer-readable program code adapted to be executed by one or more processors to implement a method comprising:

providing, at the source site, a controller, a plurality of source worker nodes, and a message queue, coupled between the controller and the plurality of source worker nodes;
providing, at the destination site, a plurality of destination worker nodes;
defining a replication context specifying a replication frequency, and information identifying objects at the source site to replicate;
identifying, by the controller, differences between a first snapshot created at the source site at a first time and a second snapshot created at the source site at a second time, after the first time;
based on the differences, generating a set of tasks comprising one or more of copying an object from the source site to the destination site, or deleting an object from the destination site;
placing, by the controller, the tasks onto the message queue;
retrieving, by a first source worker node, a first task from the message queue;
coordinating, by the first source worker node, with a first destination worker node to perform the first task;
retrieving, by a second source worker node, a second task from the message queue;
and
coordinating, by the second source worker node, with a second destination worker node to perform the second task, wherein the first snapshot comprises a first listing of objects stored in an object storage at the source site as of the first time, the second snapshot comprises a second listing of objects stored in the object storage as of the second time, and the method comprises:
receiving the first and second listings in an alphabetical sort;
comparing the alphabetically sorted first and second listings;
determining from the comparison that an object listed in both the first and second listings has a timestamp that changed between the first and second snapshots, the object listed in the first listing having an earlier timestamp, and the object listed in the second listing having a later timestamp;
and
upon the determination, generating the first task and the second task, the first task comprising an instruction to copy the object with the later timestamp to the destination site, and the second task comprising an instruction to delete the object with the earlier timestamp from the destination site.

13 more elements

providing, at the source site, a controller, a plurality of source worker nodes, and a message queue, coupled between the controller and the plurality of source worker nodes;
providing, at the destination site, a plurality of destination worker nodes;
defining a replication context specifying a replication frequency, and information identifying objects at the source site to replicate;
identifying, by the controller, differences between a first snapshot created at the source site at a first time and a second snapshot created at the source site at a second time, after the first time;
based on the differences, generating a set of tasks comprising one or more of copying an object from the source site to the destination site, or deleting an object from the destination site;
placing, by the controller, the tasks onto the message queue;
retrieving, by a first source worker node, a first task from the message queue;
coordinating, by the first source worker node, with a first destination worker node to perform the first task;
retrieving, by a second source worker node, a second task from the message queue;
and
coordinating, by the second source worker node, with a second destination worker node to perform the second task, wherein the first snapshot comprises a first listing of objects stored in an object storage at the source site as of the first time, the second snapshot comprises a second listing of objects stored in the source object storage as of the second time, and the method comprises:
receiving the first and second listings in an alphabetical sort;
comparing the alphabetically sorted first and second listings;
determining from the comparison that the first listing includes an object not included in the second listing;
and
upon the determination, generating the first task, the first task comprising an instruction to delete the object from the destination site.

13 more elements

providing, at the source site, a controller, a plurality of source worker nodes, and a message queue, coupled between the controller and the plurality of source worker nodes;
providing, at the destination site, a plurality of destination worker nodes;
defining a replication context specifying a replication frequency, and information identifying objects at the source site to replicate;
identifying, by the controller, differences between a first snapshot created at the source site at a first time and a second snapshot created at the source site at a second time, after the first time;
based on the differences, generating a set of tasks comprising one or more of copying an object from the source site to the destination site, or deleting an object from the destination site;
placing, by the controller, the tasks onto the message queue;
retrieving, by a first source worker node, a first task from the message queue;
coordinating, by the first source worker node, with a first destination worker node to perform the first task;
retrieving, by a second source worker node, a second task from the message queue;
and
coordinating, by the second source worker node, with a second destination worker node to perform the second task, wherein the first snapshot comprises a first listing of objects stored in an object storage at the source site as of the first time, the second snapshot comprises a second listing of objects stored in the source object storage as of the second time, and the method comprises:
receiving the first and second listings in an alphabetical sort;
comparing the alphabetically sorted first and second listings;
determining from the comparison that the second listing includes an object not included in the first listing;
and
upon the determination, generating the first task, the first task comprising an instruction to copy the object to the destination site.

13 more elements

Dependencies are inferred from the claim text.

Facts

Application: 15/887,563Filed 2 Feb 2018
Published: US 2019/0243547 A18 Aug 2019
Granted: 1 Sep 2020
Term ends: Not estimatedThe source records don’t support an estimate.

Classification13 codes

G06F 3/065
Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems: replication mechanisms
G06F 3/0604
Specifically adapted to achieve a particular effect: improving or facilitating administration, e.g. storage management
G06F 3/0607
Improving or facilitating administration, e.g. storage management: by facilitating the process of upgrading existing storage systems, e.g. for…
G06F 3/0613
Improving I/O performance: in relation to throughput