Scalable deduplication system and method
AssigneeGuidance Software, Inc.
InventorsShawn McCreight, Frank Lin, Prateek Arora, Kunjan Zaveri, Jason Fredrickson
Life of this patent
Estimating the term…
Abstract
A system and method for data deduplication includes a first computer device that determines duplicacy of a data item. If the data item is not a duplicate, the first computer device transmits a request to add an entry for the data item in a deduplication table of a deduplication database. The database adds the entry for the data item while enforcing uniqueness of data across one or more data fields of the deduplication table, where, in enforcing the uniqueness, the database denies an attempt by the second device to add an entry in the deduplication table for the same data item.
Claims21
In a computer system including first and second computer devices coupled to a central database device over a data communications network, a method for data deduplication comprising:
- instantiating the first computer device to a first deduplication level;
- identifying a first data item by the first computer device;
- performing by the first computer device a lookup of a deduplication table according to a duplicacy rule for determining duplicacy of the first data item at the instantiated first deduplication level, the deduplication table being stored in the central database device and including an entry for a second data item with content equal to content of the first data item, the second data item being associated with a second deduplication level different from the first deduplication level, wherein the second entry of the deduplication table stores a value corresponding to the content of the second data item;
- in response to the lookup of the lookup table based on the duplicacy rule, determining by the first computer device that there is no duplicacy of the first data item at the first deduplication level;
- in response to the determination that there is no duplicacy at the first deduplication level, transmitting by the first computer device a request to add an entry in the deduplication table for the first data item;
- and
- adding by the central database the entry for the first data item at the first deduplication level, wherein the adding includes storing a value in the deduplication table corresponding to the content of the first data item, wherein the value stored in the first entry is equal to the value stored in the second entry.
A system for data deduplication comprising:
- a server computer;
- central data store coupled to the server computer, the central data store storing a deduplication table in a deduplication database;
- and
- first and second computer devices coupled to the server computer over a data communications network,
- wherein the first computer device includes a first processor and a first memory operably coupled to the processor and storing first program instructions therein, the first processor being operable to execute the first program instructions, the first program instructions including:
- instantiating to a first deduplication level;
- identifying a first data item;
- performing a lookup of the deduplication table according to a duplicacy rule for determining duplicacy of the first data item at the instantiated first deduplication level, the deduplication table being stored in the central database device and including an entry for a second data item with content equal to content of the first data item, the second data item being associated with a second deduplication level different from the first deduplication level, wherein the second entry of the deduplication table stores a value corresponding to the content of the second data item;
- in response to the lookup of the lookup table based on the duplicacy rule, determining by the first computer device that there is no duplicacy of the first data item at the first deduplication level;
- in response the determination that there is no duplicacy at the second deduplication level, transmitting a request to the server computer over the data communications network to add an entry in the deduplication table for the first data item;
- and
- wherein the server computer includes a second processor and a second memory operably coupled to the processor and storing second program instructions therein, the second processor being operable to execute the second program instructions, the second program instructions including:
- adding the entry for the first data item in the deduplication table at the first deduplication level, wherein the adding includes storing a value in the deduplication table corresponding to the content of the first data item, wherein the value stored in the first entry is equal to the value stored in the second entry.
Dependencies are inferred from the claim text.