Data deduplication on a distributed file system using conditional writes
AssigneeEMC IP Holding Company LLC
InventorRajiv Desai
Life of this patent
Estimating the term…
Abstract
A write request is transmitted from a client to a metadata server (“MDS”), wherein the write request comprises an object identifier associated with a data object. An object store location for an object store is received from the MDS. A conditional write request is transmitted to the object store using the object store location, wherein the conditional write request includes the object identifier and the data object. A metadata response is received from the object store. The metadata response is determined to contain an object designator. A commit request is transmitted to the MDS, wherein the commit response includes the object designator.
Claims12
A method for deduplicating data on a distributed file system, the method comprising:
- transmitting, by a client, a write request to a metadata server (“MDS”) associated with the distributed file system, wherein the MDS maintains location data for all data objects in the distributed file system, wherein the write request comprises an object identifier associated with a data object;
- receiving, by the client, an object store location for an object store from the MDS;
- transmitting, by the client, a metadata request to the object store using the object store location prior to transmitting the data object, wherein the metadata request includes the object identifier, and wherein the object store is separate from the MDS;
- receiving, by the client, a metadata response from the object store to the metadata request;
- determining, by the client, that the object store contains the data object when the metadata response includes an object designator and determining that the object store does not contain the data object when the metadata response does not include the object designator, where the data object is transmitted to the object store and stored at the object store when the metadata response does not include the object designator;
- and
- transmitting, by the client, a commit request to the MDS, wherein the commit response includes the object designator, wherein the object identifier uniquely identifies the data object and the object designator allows the MDS to track how many instances of the data object are stored in the distributed file system.
A non-transitory computer readable storage medium comprising processor instructions for deduplicating data on a distributed file system, the instructions comprising:
- transmitting, by a client, a write request to a metadata server (“MDS”) associated with the distributed file system, wherein the MDS maintains location data for all data objects in the distributed file system, wherein the write request comprises an object identifier associated with a data object;
- receiving, by the client, an object store location for an object store from the MDS;
- transmitting a metadata request to the object store using the object store location prior to transmitting the data object, wherein the metadata request includes the object identifier, and wherein the object store is separate from the MDS;
- receiving, by the client, a metadata response from the object store to the metadata request;
- determining, by the client, that the object store contains the data object when the metadata response includes an object designator and determining that the object store does not contain the data object when the metadata response does not include the object designator, where the data object is transmitted to the object store and stored at the object store when the metadata response does not include the object designator;
- and
- transmitting, by the client, a commit request to the MDS, wherein the commit response includes the object designator, wherein the object identifier uniquely identifies the data object and the object designator allows the MDS to track how many instances of the data object are stored in the distributed file system.
A system for deduplicating data on a distributed file system, the system comprising a non-transitory computer readable medium and processor enabled to execute instructions for:
- transmitting, by a client, a write request to a metadata server (“MDS”) associated with the distributed file system, wherein the MDS maintains location data for all data objects in the distributed file system, wherein the write request comprises an object identifier associated with a data object;
- receiving, by the client, an object store location for an object store from the MDS;
- transmitting a metadata request to the object store using the object store location prior to transmitting the data object, wherein the metadata request includes the object identifier, and wherein the object store is separate from the MDS;
- receiving, by the client, a metadata response from the object store to the metadata request;
- determining, by the client, that the object store contains the data object when the metadata response includes an object designator and determining that the object store does not contain the data object when the metadata response does not include the object designator, where the data object is transmitted to the object store and stored at the object store when the metadata response does not include the object designator;
- and
- transmitting, by the client, a commit request to the MDS, wherein the commit response includes the object designator, wherein the object identifier uniquely identifies the data object and the object designator allows the MDS to track how many instances of the data object are stored in the distributed file system.
Dependencies are inferred from the claim text.