# US 2015/0106345 A1 · Multi-node hybrid deduplication

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2015/0106345 A1
- Document ID: US20150106345A1
- Status: Published application
- Kind: A1
- Application number: 14/515,020
- Filed: 2014-10-15
- Published: 2015-04-16
- Earliest claimed: 2013-10-15
- Assignees: Hitachi Data Systems Corporation
- Applicants: Sepaton, Inc.
- Inventors: Ronald Ray Trimble; Jeffrey V. Tofano; Thomas R. Ramsdell; Jon Christopher Kennedy
- Web page: https://patentgrep.com/patent/US20150106345A1

## CPC

- G06F 16/1748: {De-duplication implemented within the file system, e.g. based on file segments (de-duplication techniques in storage systems for the management of data blocks G06F3/0641)}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

According to at least one embodiment, a data storage system is provided. The data storage system includes memory, at least one processor in data communication with the memory, and a deduplication director component executable by the at least one processor. The deduplication director component is configured to receive data for storage on the data storage system, analyze the data to determine whether the data is suitable for at least one of summary-based deduplication, content-based deduplication, and no deduplication, and store, in a common object store, at least one of the data and a reference to duplicate data stored in the common object store.

## Claims

### Claim 1

A data storage system comprising:
memory;
at least one processor in data communication with the memory; and
a deduplication director component executable by the at least one processor and configured to:
receive data for storage on the data storage system;
analyze the data to determine whether the data is suitable for at least one of summary-based deduplication, content-based deduplication, and no deduplication; and
store, in a common object store, at least one of the data and a reference to duplicate data stored in the common object store.

### Claim 2 (depends on 1)

The data storage system of claim 1, wherein the deduplication director component is configured to store, in the common object store, the data in response to determining that the data is suitable for no deduplication.

### Claim 3 (depends on 1)

The data storage system of claim 1, wherein the deduplication director component is configured to store, in the common object store, the reference to the duplicate data at least in part by executing a summary-based deduplication component in response to determining that the data is suitable for summary-based deduplication, the summary-based deduplication component being configured to:
generate a summary of the data;
identify the data as being a copy of the duplicate data at least in part by comparing the summary to a duplicate summary of the duplicate data; and
store, in the common object store, the reference to the duplicate data in response to identifying the data as being a copy of the duplicate data.

### Claim 4 (depends on 1)

The data storage system of claim 1, wherein the deduplication director component is configured to store, in the common object store, the reference to the duplicate data at least in part by executing a content-based deduplication component in response to determining that the data is suitable for content-based deduplication, the content-based deduplication component being configured to:
identify the data as being a copy of the duplicate data by comparing the data to the duplicate data; and
store, in the common object store, the reference to the duplicate data in response to identifying the data as being a copy of the duplicate data.

### Claim 5 (depends on 1)

The data storage system of claim 1, wherein the deduplication director component is configured to analyze the data at least in part by identifying at least one deduplication template to apply to the data, the at least one deduplication template specifying one or more deduplication options for the data.

### Claim 6 (depends on 5)

The data storage system of claim 5, wherein the at least one deduplication template specifies that at least one of a summary-based deduplication component and a content-based deduplication component process the data.

### Claim 7 (depends on 1)

The data storage system of claim 1, further comprising a content-based deduplication component and a summary-based deduplication component, wherein the common object store stores a map that associates the data with one or more objects stored in the common object store and at least one of the content-based deduplication component and the summary-based deduplication component is configured to deduplicate the map.

### Claim 8 (depends on 7)

The data storage system of claim 7, wherein the content-based deduplication component is configured to deduplicate the map at least in part by re-segmenting at least one data object associated with the map.

### Claim 9 (depends on 1)

The data storage system of claim 1, further comprising:
a file system interface component configured to receive one or more files for storage on the data storage system; and
a common object store component executable by the at least one processor and configured to store each of the one or more files as one or more data objects structured differently from the one or more files.

### Claim 10 (depends on 9)

The data storage system of claim 9, wherein the file system interface component includes a proxy file system interface configured to receive source level deduplication information.

### Claim 11 (depends on 10)

The data storage system of claim 10, wherein the proxy file system interface is further configured to determine whether subject data identified within the source level deduplication information is stored in the data storage system.

### Claim 12

A method of deduplicating data using a data storage system, the method comprising:
receiving, by the data storage system, data for storage on the data storage system;
analyzing the data to determine whether the data is suitable for at least one of summary-based deduplication, content-based deduplication, and no deduplication; and
storing, in a common object store, at least one of the data and a reference to duplicate data stored in the common object store.

### Claim 13 (depends on 12)

The method of claim 12, wherein analyzing the data includes identifying at least one deduplication template to apply to the data, the at least one deduplication template specifying one or more deduplication options for the data.

### Claim 14 (depends on 13)

The method of claim 13, wherein identifying the at least one deduplication template includes identifying at least one deduplication template that specifies at least one of a summary-based deduplication component and a content-based deduplication component process the data.

### Claim 15 (depends on 12)

The method of claim 12, wherein the common object store stores a map that associates the data with one or more objects stored in the common object store and the method further comprises deduplicating the map.

### Claim 16 (depends on 15)

The method of claim 15, wherein deduplicating the map includes re-segmenting at least one data object associated with the map.

### Claim 17 (depends on 12)

The method of claim 12, further comprising:
receiving one or more files for storage on the data storage system; and
storing each of the one or more files as one or more data objects structured differently from the one or more files.

### Claim 18 (depends on 17)

The method of claim 17, further comprising receiving source level deduplication information.

### Claim 19 (depends on 18)

The method of claim 18, further comprising determining whether subject data identified within the source level deduplication information is stored in the data storage system.

### Claim 20

A non-transitory computer readable medium storing computer executable instructions configured to instruct at least one processor to execute a hybrid deduplication process, the computer executable instructions including instructions to:
receive data for storage on the data storage system;
analyze the data to determine whether the data is suitable for at least one of summary-based deduplication, content-based deduplication, and no deduplication; and
store, in a common object store, at least one of the data and a reference to duplicate data stored in the common object store.
