Partitioning of deduplication domains in storage systems
AssigneeInternational Business Machines Corporation
InventorsMiles Mulholland, Alex Dicks, Dominic Tomkins, Eric John Bartlett
Life of this patent
Estimating the term…
Abstract
Method and system for partitioning of deduplication domains in storage systems. The method includes constructing a data structure having multiple nodes representing data chunks and edges between the nodes representing a weighting of deduplication references between the data chunks, and performing clustering of the nodes of the data structure to split the nodes into clusters of tightly related nodes based on the weightings of the edges. The data chunks represented by a cluster of nodes are migrated to a deduplication domain to restrict deduplication to between only the data chunks in the deduplication domain.
Claims25
A computer-implemented method for partitioning of deduplication domains in a storage system, comprising:
- constructing a data structure having multiple nodes representing data chunks and edges between the nodes representing a weighting of deduplication references between the data chunks, wherein constructing the data structure includes:
- adding a new node and a new edge when a new deduplication reference is created with a new data chunk,
- incrementing edge weightings when a new deduplication reference is created between existing data chunks, and
- decrementing edge weightings when a deduplication reference is removed between existing data chunks;
- and
- performing clustering of the nodes of the data structure to split the nodes into clusters of tightly related nodes based on the weightings of the edges;
- and
- migrating the data chunks represented by a cluster of nodes to a deduplication domain to restrict deduplication to between only the data chunks in the deduplication domain.
A computer-implemented method for partitioning of deduplication domains in storage systems, comprising:
- providing a staging area for new data chunks including a reference structure of possible deduplication domains into which a new data chunk may be migrated;
- carrying out a sampling period for a new data chunk to compare input/output operations for the new data chunk to the reference structure for the deduplication domains;
- selecting a deduplication domain for the new data chunk based on a number of hits to the reference structure indicating deduplication references;
- and
- migrating the new data chunk to a core storage controller of the selected deduplication domain.
A system for partitioning of deduplication domains in storage systems, comprising:
- a processor and a memory configured to provide computer program instructions to the processor to execute a function of the following components:
- a data structure component for constructing a data structure having multiple nodes representing data chunks and edges between the nodes representing a weighting of deduplication references between the data chunks,
- wherein constructing the data structure includes:
- incrementing edge weightings when a new deduplication reference is created between existing data chunks, and
- decrementing edge weightings when a deduplication reference is removed between existing data chunks;
- a clustering component for performing clustering of the nodes of the data structure to split the nodes into clusters of tightly related nodes based on the weightings of the edges;
- and
- a migrating component for migrating the data chunks represented by a cluster of nodes to a deduplication domain to restrict deduplication to between only the data chunks in the deduplication domain.
A system for partitioning of deduplication domains in storage systems, comprising:
- a processor and a memory configured to provide computer program instructions to the processor to execute a function of the following components:
- a staging component for providing a staging area for new data chunks including a reference structure of possible deduplication domains into which a new data chunk may be migrated;
- a sampling component for carrying out a sampling period for a new data chunk to compare input/output operations for the new data chunk to the reference structure for the deduplication domains,
- wherein no deduplications are attempted during the sampling period;
- and
- a domain selection component for selecting a deduplication domain for the new data chunk based on a number of hits to the reference structure indicating deduplication references,
- wherein the staging component is a staging storage controller serving input/output operations for a data chunk during the sampling period in which no deduplication takes place,
- wherein the possible deduplication domains are each provided by an associated core storage controller having a deduplication database, and the reference structure is populated from the deduplication databases of the core storage controllers.
A computer program product for partitioning of deduplication domains in storage systems, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to:
- construct a data structure having multiple nodes representing data chunks and edges between the nodes representing a weighting of deduplication references between the data chunks,
- wherein edge weightings are adjusted in response to a new deduplication reference being created between existing data chunks,
- wherein the data chunks are blocks of data;
- perform clustering of the nodes of the data structure to split the nodes into clusters of tightly related nodes based on the weightings of the edges;
- and
- migrate the data chunks represented by a cluster of nodes to a deduplication domain to restrict deduplication to between only the data chunks in the deduplication domain.
Dependencies are inferred from the claim text.