Back to results
US 11,599,505 B2Granted
Reference set construction for data deduplication
AssigneeWestern Digital Technologies, Inc.
InventorsAshish Singhai, Ashwin Narasimha, Vijay Karamcheti, Tanay Goel
Life of this patent
Estimating the term…
Abstract
By way of example, a data storage system may comprise, a non-transitory storage device storing data blocks in chunks, and a storage logic coupled to the non-transitory storage device that manages storage of data on the storage device. The storage logic is executable to receive a data stream for storage in a non-transitory storage device, the data stream including one or more data blocks, analyze the data stream to determine a domain, retrieve a pre-configured reference set based on the domain, and deduplicate the one or more data blocks of the data stream using the pre-configured reference set.
Claims19
A data storage system, comprising:
- a non-transitory storage device storing data blocks in chunks;
- a processor coupled to the non-transitory storage device, the processor configured to:
- receive a data stream for storage in the non-transitory storage device, the data stream including one or more data blocks;
- analyze the data stream to determine a domain;
- retrieve a pre-configured reference set based on the domain, the pre-configured reference set defined by a group of data blocks;
- and
- deduplicate the one or more data blocks of the data stream using the pre-configured reference set by being further configured to:
- encode, responsive to the one or more blocks in the data stream and the group of data blocks in the reference set meeting a similarity criterion, the one or more blocks based on the reference set,
- aggregate, responsive to:
- (a) the one or more blocks in the data stream and the group of data blocks in the reference set not meeting the similarity criterion, and
- (b) the one or more blocks in the data stream having a threshold similarity to each other,
- the one or more blocks into an aggregated set of data blocks comprising data blocks different from the data blocks in the pre-configured reference set, and
- generate a new reference set using the set of data blocks.
10 more elements
A method comprising:
- receiving a data stream for storage in a non-transitory storage device, the data stream including one or more data blocks;
- analyzing the data stream to determine a domain;
- retrieving a pre-configured reference set based on the domain, the pre-configured reference set defined by a group of data blocks;
- and
- deduplicating the one or more data blocks of the data stream using the pre-configured reference set, the deduplicating comprising:
- encoding, responsive to the one or more blocks in the data stream and the group of data blocks in the reference set meeting a similarity criterion, the one or more blocks based on the reference set,
- aggregating, responsive to:
- (a) the one or more blocks in the data stream and the group of data blocks in the reference set not meeting the similarity criterion, and
- (b) the one or more blocks in the data stream having a threshold similarity to each other,
- the one or more blocks into an aggregated set of data blocks comprising data blocks different from the data blocks in the pre-configured reference set;
- and
- generating a new reference set using the aggregated set of data blocks.
9 more elements
A data storage system comprising:
- means for receiving a data stream for storage in a non-transitory storage device, the data stream including one or more data blocks;
- means for analyzing the data stream to determine a domain;
- means for retrieving a pre-configured reference set based on the domain, the pre-configured reference set defined by a group of data blocks;
- and
- means for deduplicating the one or more data blocks of the data stream using the pre-configured reference set, the means for deduplicating comprising:
- means for encoding, responsive to the one or more blocks in the data stream and the group of data blocks in the reference set meeting a similarity criterion, the one or more blocks based on the reference set,
- means for aggregating, responsive to:
- (a) the one or more blocks in the data stream and the group of data blocks in the reference set not meeting the similarity criterion, and
- (b) the one or more blocks in the data stream having a threshold similarity to each other,
- the one or more blocks into an aggregated set of data blocks comprising data blocks different from the data blocks in the pre-configured reference set, and
- means for generating a new reference set using the set of data blocks.
8 more elements
Dependencies are inferred from the claim text.