Back to results
US 2021/0279354 A1Published application
Live data conversion and migration for distributed data object systems
AssigneeDropbox, Inc.
InventorsAshish Gandhi, Renjish Abraham, Kevin Farrell
Life of this patent
Estimating the term…
Abstract
Techniques are disclosed for migrating data objects stored by the source DDOS from the source DDOS to the target DDOS while at least the source DDOS is live and available to process requests for access to the data objects being migrated. The techniques also provide eventual consistency between data objects that are created, updated, or deleted in the source DDOS that are applicable to the migration and that occur while the migration is being performed.
Claims20
A computer-implemented method comprising:
- using a set of long-lived worker processes to read a first set of data objects from a source distributed data object system and to write a second set of data objects corresponding to the first set of data objects to a target distributed data object system;
- wherein each worker process of the set of long-lived worker processes reads at least two data objects of the first set of data objects from the source distributed data object system and writes at least two data objects of the second set of data objects to the target distributed data object system;
- comparing a first set of signatures for the first set of data objects to a second set of signatures for the second set of data objects;
- and
- based on a set of mismatches between the first set of signatures and the second set of signatures, using the set of long-lived worker processes to read a first set of modified data objects of the first set of data objects corresponding to the set of mismatches and to write a second set of modified data objects corresponding to the first set of modified data objects to the target distributed data object system.
2 more elements
One or more non-transitory storage media storing instructions which, when executed by a computing system having one or more processors, cause the computing system to perform:
- using a set of long-lived worker processes to stream a first set of data objects from a source distributed data object system and to a target distributed data object system for storage in the target distributed data object system as a second set of data objects corresponding to the first set of data objects;
- wherein each worker process of the set of long-lived worker processes streams at least two data objects of the first set of data objects from the source distributed data object system to the target distributed data object system;
- comparing a first set of signatures for the first set of data objects to a second set of signatures for the second set of data objects;
- and
- based on a set of mismatches between the first set of signatures and the second set of signatures, using the set of long-lived worker processes to stream a first set of modified data objects of the first set of data objects corresponding to the set of mismatches and to the target distributed data object system for storage for storage in the target distributed data object system as a second set of modified data objects corresponding to the first set of modified data objects.
2 more elements
A computing system comprising:
- a plurality of processors;
- storage media;
- and
- instructions stored in the storage media which, when executed by the plurality of processors, cause the computing system to perform:
- using a set of long-lived worker processes to read a first set of data objects from a source distributed data object system and to write a second set of data objects corresponding to the first set of data objects to a target distributed data object system;
- wherein each worker process of the set of long-lived worker processes reads at least two data objects of the first set of data objects from the source distributed data object system and writes at least two data objects of the second set of data objects to the target distributed data object system;
- comparing a first set of signatures for the first set of data objects to a second set of signatures for the second set of data objects;
- and
- based on a set of mismatches between the first set of signatures and the second set of signatures, using a set of long-lived worker processes to read a first set of modified data objects of the first set of data objects corresponding to the set of mismatches and to write a second set of modified data objects corresponding to the first set of modified data objects to the target distributed data object system.
6 more elements
Dependencies are inferred from the claim text.