# US 2019/0245918 A1 · Distributed replication of an object

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0245918 A1
- Document ID: US20190245918A1
- Status: Published application
- Kind: A1
- Application number: 15/887,680
- Filed: 2018-02-02
- Published: 2019-08-08
- Earliest claimed: 2018-02-02
- Assignees: EMC IP Holding Company LLC
- Applicants: EMC IP Holding Company LLC
- Inventors: Kevin Xu; Abhinav Duggal; Atul Avinash Karmarkar; Philip Shilane
- Web page: https://patentgrep.com/patent/US20190245918A1

## CPC

- H04L 67/1095: Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
- G06F 16/1744: {using compression, e.g. sparse files}
- G06F 16/1748: {De-duplication implemented within the file system, e.g. based on file segments (de-duplication techniques in storage systems for the management of data blocks G06F3/0641)}
- G06F 16/178: Techniques for file synchronisation in file systems
- H04L 63/08: {for authentication of entities (cryptographic mechanisms or cryptographic arrangements for entity authentication H04L9/32)}
- H04L 63/0869: {for achieving mutual authentication (cryptographic mechanisms or cryptographic arrangements for mutual authentication H04L9/3273)}
- H04L 67/06: specially adapted for file transfer, e.g. file transfer protocol [FTP]
- H04L 67/1001: for accessing one among a plurality of replicated servers
- H04L 67/1097: for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A source worker node at a source site fetches a task from a message queue. The task specifies replicating a first object at the source site to a destination site. A request for a connection is issued from the source worker node to the destination site. The request is received by a load balancer at the destination site and assigned to a destination worker node. A connection is established between the source and destination worker nodes. A determination is made that the destination site does not include an object that is the same as the first object. Upon the determination, a deduplication is performed between the source and destination worker nodes of segments into which the first object has been divided. Deduplicated segments of the first object are transmitted from the source worker node to the destination worker node for storage at the destination site.

## Claims

### Claim 1

A method comprising:
fetching, by a first source worker node at a source site, a first task from a message queue, the first task specifying replicating a first object at the source site to a destination site;
issuing, from the first source worker node to the destination site, a first request for a first connection, the first request being received by a load balancer at the destination site and assigned to a first destination worker node;
establishing the first connection between the first source and destination worker nodes;
determining that the destination site does not include an object having a same name, timestamp, size, and checksum as the first object;
upon the determining, performing between the first source worker node and the first destination worker node a deduplication of segments into which the first object has been divided; and
transmitting from the first source worker node to the first destination worker node the deduplicated segments of the first object for storage at the destination site.

### Claim 2 (depends on 1)

The method of claim 1 comprising:
fetching, by a second source worker node at the source site, a second task from the message queue, the second task specifying deleting a second object from the destination site;
issuing, from the second source worker node to the destination site, a second request for a second connection, the second requested being received by the load balancer and assigned to a second destination worker node;
establishing the second connection between the second source and destination worker nodes; and
after the establishing the second connection, instructing the second destination worker node to delete the second object.

### Claim 3 (depends on 1)

The method of claim 1 comprising:
receiving, at a client, an instruction to replicate a file on the client to the destination site;
issuing, from the client to the destination site, a second request for a second connection, the second request being received by the load balancer at the destination site and assigned to a second destination worker node;
establishing the second connection between the client and second destination worker node;
performing between the client and the second destination worker node a deduplication of segments into which the file has been divided; and
transmitting from the client to the second destination worker node the deduplicated segments of the file for storage at the destination site.

### Claim 4 (depends on 1)

The method of claim 1 wherein the establishing the first connection between the first source and destination worker nodes comprises:
transmitting an authentication key from the first source worker node to the first destination worker node to authenticate the first source worker node.

### Claim 5 (depends on 1)

The method of claim 1 comprising:
transmitting, from the first source worker node to the first destination worker node, a first slice recipe, the first slice recipe comprising a first set of fingerprints corresponding to a first set of segments of the first object;
receiving, from the first destination worker node, an identification of fingerprints from the first set of fingerprints corresponding to segments of the first set of segments that are missing at the destination site;
compressing the missing segments of the first set of segments into a first compression region;
transmitting the first compression region to the first destination worker node;
transmitting, from the first source worker node to the first destination worker node, a second slice recipe, the second slice recipe comprising a second set of fingerprints corresponding to a second set of segments of the first object;
receiving, from the first destination worker node, an identification of fingerprints from the second set of fingerprints corresponding to segments of the second set of segments that are missing at the destination site;
compressing the missing segments of the second set of segments into a second compression region; and
transmitting the second compression region to the first destination worker node.

### Claim 6 (depends on 5)

The method of claim 5 wherein the first and second slice recipes are organized into a similarity group, and slice recipes outside the similarity group are not considered when performing the deduplication of the first object.

### Claim 7

A system comprising:
a processor-based system executed on a computer system and configured to:
fetch, by a first source worker node at a source site, a first task from a message queue, the first task specifying replicating a first object at the source site to a destination site;
issue, from the first source worker node to the destination site, a first request for a first connection, the first request being received by a load balancer at the destination site and assigned to a first destination worker node;
establish the first connection between the first source and destination worker nodes;
determine that the destination site does not include an object having a same name, timestamp, size, and checksum as the first object;
upon the determination, perform between the first source worker node and the first destination worker node a deduplication of segments into which the first object has been divided; and
transmit from the first source worker node to the first destination worker node the deduplicated segments of the first object for storage at the destination site.

### Claim 8 (depends on 7)

The system of claim 7 wherein the processor-based system is configured to:
fetch, by a second source worker node at the source site, a second task from the message queue, the second task specifying deleting a second object from the destination site;
issue, from the second source worker node to the destination site, a second request for a second connection, the second requested being received by the load balancer and assigned to a second destination worker node;
establish the second connection between the second source and destination worker nodes; and
after the establishing the second connection, instruct the second destination worker node to delete the second object.

### Claim 9 (depends on 7)

The system of claim 7 wherein the processor-based system is configured to:
receive, at a client, an instruction to replicate a file on the client to the destination site;
issue, from the client to the destination site, a second request for a second connection, the second request being received by the load balancer at the destination site and assigned to a second destination worker node;
establish the second connection between the client and second destination worker node;
perform between the client and the second destination worker node a deduplication of segments into which the file has been divided; and
transmit from the client to the second destination worker node the deduplicated segments of the file for storage at the destination site.

### Claim 10 (depends on 7)

The system of claim 7 wherein the processor-based system is configured to:
transmit an authentication key from the first source worker node to the first destination worker node to authenticate the first source worker node.

### Claim 11 (depends on 7)

The system of claim 7 wherein the processor-based system is configured to:
transmit, from the first source worker node to the first destination worker node, a first slice recipe, the first slice recipe comprising a first set of fingerprints corresponding to a first set of segments of the first object;
receive, from the first destination worker node, an identification of fingerprints from the first set of fingerprints corresponding to segments of the first set of segments that are missing at the destination site;
compress the missing segments of the first set of segments into a first compression region;
transmit the first compression region to the first destination worker node;
transmit, from the first source worker node to the first destination worker node, a second slice recipe, the second slice recipe comprising a second set of fingerprints corresponding to a second set of segments of the first object;
receive, from the first destination worker node, an identification of fingerprints from the second set of fingerprints corresponding to segments of the second set of segments that are missing at the destination site;
compress the missing segments of the second set of segments into a second compression region; and
transmit the second compression region to the first destination worker node.

### Claim 12 (depends on 11)

The system of claim 11 wherein the first and second slice recipes are organized into a similarity group, and slice recipes outside the similarity group are not considered when performing the deduplication of the first object.

### Claim 13

A computer program product comprising a non-transitory computer-readable medium having a computer-readable program code embodied therein, the computer-readable program code adapted to be executed by one or more processors to implement a method comprising:
fetching, by a first source worker node at a source site, a first task from a message queue, the first task specifying replicating a first object at the source site to a destination site;
issuing, from the first source worker node to the destination site, a first request for a first connection, the first request being received by a load balancer at the destination site and assigned to a first destination worker node;
establishing the first connection between the first source and destination worker nodes;
determining that the destination site does not include an object having a same name, timestamp, size, and checksum as the first object;
upon the determining, performing between the first source worker node and the first destination worker node a deduplication of segments into which the first object has been divided; and
transmitting from the first source worker node to the first destination worker node the deduplicated segments of the first object for storage at the destination site.

### Claim 14 (depends on 13)

The computer program product of claim 13 wherein the method comprises:
fetching, by a second source worker node at the source site, a second task from the message queue, the second task specifying deleting a second object from the destination site;
issuing, from the second source worker node to the destination site, a second request for a second connection, the second requested being received by the load balancer and assigned to a second destination worker node;
establishing the second connection between the second source and destination worker nodes; and
after the establishing the second connection, instructing the second destination worker node to delete the second object.

### Claim 15 (depends on 13)

The computer program product of claim 13 wherein the method comprises:
receiving, at a client, an instruction to replicate a file on the client to the destination site;
issuing, from the client to the destination site, a second request for a second connection, the second request being received by the load balancer at the destination site and assigned to a second destination worker node;
establishing the second connection between the client and second destination worker node;
performing between the client and the second destination worker node a deduplication of segments into which the file has been divided; and
transmitting from the client to the second destination worker node the deduplicated segments of the file for storage at the destination site.

### Claim 16 (depends on 13)

The computer program product of claim 13 wherein the establishing the first connection between the first source and destination worker nodes comprises:
transmitting an authentication key from the first source worker node to the first destination worker node to authenticate the first source worker node.

### Claim 17 (depends on 13)

The computer program product of claim 13 wherein the method comprises:
transmitting, from the first source worker node to the first destination worker node, a first slice recipe, the first slice recipe comprising a first set of fingerprints corresponding to a first set of segments of the first object;
receiving, from the first destination worker node, an identification of fingerprints from the first set of fingerprints corresponding to segments of the first set of segments that are missing at the destination site;
compressing the missing segments of the first set of segments into a first compression region;
transmitting the first compression region to the first destination worker node;
transmitting, from the first source worker node to the first destination worker node, a second slice recipe, the second slice recipe comprising a second set of fingerprints corresponding to a second set of segments of the first object;
receiving, from the first destination worker node, an identification of fingerprints from the second set of fingerprints corresponding to segments of the second set of segments that are missing at the destination site;
compressing the missing segments of the second set of segments into a second compression region; and
transmitting the second compression region to the first destination worker node.

### Claim 18 (depends on 17)

The computer program product of claim 17 wherein the first and second slice recipes are organized into a similarity group, and slice recipes outside the similarity group are not considered when performing the deduplication of the first object.
