Technique for global deduplication across datacenters with minimal coordination
AssigneeNetApp, Inc.
InventorsKiran Nenmeli Srinivasan, Kishore Kasi Udayashankar, Swetha Krishnan
Life of this patent
Estimating the term…
Abstract
A system and method for global data de-duplication in a cloud storage environment utilizing a plurality of data centers is provided. Each cloud storage gateway appliance divides a data stream into a plurality of data objects and generates a content-based hash value as a key for each data object. An IMMUTABLE PUT operation is utilized to store the data object at the associated key within the cloud.
Claims20
A method for de-duplicating a data stream, the method comprising:
- chunking the data stream into a plurality of data objects;
- calculating a key associated with each of the data objects;
- performing an immutable put operation to store each data object in a cloud storage environment, each data object indexed by its associated calculated key;
- and
- updating metadata associated with the data stream to enable subsequent retrieval of the data objects from the cloud storage environment, whereby communication among a plurality of data centers storing data in the cloud storage environment is obviated.
A system for de-duplicating a data stream, the system comprising:
- a cloud storage gateway appliance operatively interconnected with a cloud storage system via a network;
- one or more clients operatively interconnected with the cloud storage gateway appliance, at least one of the one or more clients configured to forward the data stream to the cloud storage gateway appliance;
- wherein the cloud storage gateway appliance is configured to chunk the data stream into a plurality of data objects, compute a key for each of the data objects and execute an immutable put operation to store each data object with its associated key in the cloud storage system.
A non-transitory computer readable medium, the non-transitory computer readable medium comprising:
- program instructions for chunking the data stream into a plurality of data objects;
- program instructions for calculating a key associated with each of the data objects;
- program instructions for performing an immutable put operation to store each data object in a cloud storage environment, each data object indexed by its associated calculated key;
- and
- program instructions for updating metadata associated with the data stream to enable subsequent retrieval of the data objects from the cloud storage environment.
Dependencies are inferred from the claim text.