# US 8,280,926 B2 · Scalable de-duplication mechanism

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 8,280,926 B2
- Document ID: US8280926B2
- Status: Granted
- Kind: B2
- Application number: 12/355,188
- Filed: 2009-01-16
- Published: 2009-07-16
- Granted: 2012-10-02
- Earliest claimed: 2003-08-05
- Assignees: Sepaton, Inc.
- Inventors: Miklos Sandorfi; Timmie G. Reiter
- Web page: https://patentgrep.com/patent/US8280926B2

## CPC

- G06F 11/1004: {to protect a block of data words, e.g. CRC or checksum (G06F11/1076 takes precedence; security arrangements for protecting computers or computer systems against unauthorized activity G06F21/00)}
- G06F 11/1453: {using de-duplication of the data}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A method for removing redundant data from a backup storage system is presented. In one example, the method may include receiving the application layer data object, selecting a de-duplication domain from a plurality of de-duplication domains based at least in part on a data object characteristic associated with the de-duplication domain, determining that the application layer data object has the characteristic and directing the application layer data object to the de-duplication domain.

## Claims

### Claim 1

A computerized method for classifying data from a backup application for de-duplication of redundant data based on one or more characteristics of the data, the method comprising:
receiving, by a computing device, a backup data object from a backup application, wherein the backup application created the backup data object, and a data object characteristic of the backup data object is not known to the computing device;
determining the data object characteristic of the backup data object based on metadata associated with the backup data object;
selecting a de-duplication domain from a plurality of de-duplication domains based at least in part on the data object characteristic of the backup data object, wherein each de-duplication domain from the plurality of de-duplication domains is associated with:
a data object characteristic that defines a data object type;
a de-duplication method selected from a plurality of de-duplication methods that is tailored to de-duplicate redundant data among backup data objects of the data object type, wherein backup data objects of the data object type have redundant data because they share the same data object characteristic; and
a data object type that is different than each data object type associated with the remaining de-duplication domains from the plurality of de-duplication domains; and
directing the backup data object to the selected de-duplication domain for de-duplication, wherein the selected de-duplication domain executes the de-duplication method that is associated with the selected de-duplication domain to de-duplicate redundant data among the backup data object, and one or more other backup data objects also directed to the selected de-duplication domain, based on the data object characteristic associated with the selected de-duplication domain.

### Claim 2 (depends on 1)

The method according to claim 1, wherein receiving the backup data object comprises:
receiving a data stream; and
identifying the backup data object using metadata included in the data stream.

### Claim 3 (depends on 2)

The method according to claim 2, wherein receiving the data stream includes receiving a multiplexed data stream.

### Claim 4 (depends on 2)

The method according to claim 2, further comprising extracting metadata included in the data stream with the backup data object.

### Claim 5 (depends on 4)

The method according to claim 4, wherein selecting the de-duplication domain from the plurality of de-duplication domains includes comparing the extracted metadata associated with the backup data object to each set of data object characteristics associated with each de-duplication domain.

### Claim 6 (depends on 4)

The method according to claim 4, wherein extracting the metadata included in the data stream includes extracting at least one of a back-up policy name, a data source type, a data source name, a back-up application name, an operating system type, a data type, a back-up type, a filename, a directory structure and chronological information.

### Claim 7 (depends on 1)

The method according to claim 1, further comprising configuring each of the plurality of de-duplication domains to use one of a plurality of de-duplication methods.

### Claim 8 (depends on 7)

The method according to claim 7, wherein configuring each of the plurality of de-duplication domains includes configuring each of the plurality of de-duplication domains to use one de-duplication method selected from the group comprising hash-fingerprinting, pattern recognition and content aware de-duplication.

### Claim 9 (depends on 1)

The method according to claim 1, further comprising:
de-duplicating, within the selected de-duplication domain, the backup data object; and
adjusting the data object characteristic associated with at least one of the plurality of de-duplication domains based on a result of the act of de-duplicating.

### Claim 10 (depends on 9)

The method according to claim 9, wherein adjusting the data object characteristic includes storing data in a de-duplication domain database.

### Claim 11 (depends on 1)

The method according to claim 1, wherein the method is executed in a grid computing environment.

### Claim 12 (depends on 1)

The method according to claim 1, wherein the method is executed on a back-up storage system while data is not being backed-up to the back-up storage system.

### Claim 13 (depends on 1)

The method according to claim 1, wherein the method is executed on a back-up storage system while data is being backed-up to the back-up storage system.

### Claim 14 (depends on 1)

The method of claim 1, wherein directing the backup data object to the selected de-duplication domain for de-duplication comprises providing a reference to the backup data object to the selected de-duplication domain instead of providing a copy of the data object to the de-duplication domain.

### Claim 15

A computer program product, tangibly embodied in a non-transitory computer readable medium, the computer program product including instructions being configured to cause a data processing apparatus to:
receive a backup data object from a backup application, wherein the backup application created the backup data object, and a data object characteristic of the backup data object is not known to the computing device;
determine the data object characteristic of the backup data object based on metadata associated with the backup data object;
select a de-duplication domain from a plurality of de-duplication domains based at least in part on the data object characteristic of the backup data object, wherein each de-duplication domain from the plurality of de-duplication domains is associated with
a data object characteristic that defines a data object type;
a de-duplication method selected from a plurality of de-duplication methods that is tailored to de-duplicate redundant data among backup data objects of the data object type, wherein backup data objects of the data object type have redundant data because they share the same data object characteristic; and
a data object type that is different than each data object type associated with the remaining de-duplication domains from the plurality of de-duplication domains; and
direct the backup data object to the selected de-duplication domain for de-duplication, wherein the selected de-duplication domain executes the de-duplication method that is associated with the selected de-duplication domain to de-duplicate redundant data among the backup data object, and one or more other backup data objects also directed to the selected de-duplication domain, based on the data object characteristic associated with the selected de-duplication domain.

### Claim 16

A system for classifying data from a backup application for de-duplication of redundant data based on one or more characteristics of the data, the system comprising:
a plurality of de-duplication domains, wherein each de-duplication domain from the plurality of de-duplication domains is associated with
a data object characteristic that defines a data object type;
a de-duplication method selected from a plurality of de-duplication methods that is tailored to de-duplicate redundant data among backup data objects of the data object type, wherein backup data objects of the data object type have redundant data because they share the same data object characteristic; and
a data object type that is different than each data object type associated with the remaining de-duplication domains from the plurality of de-duplication domains; and
a controller coupled to the plurality of de-duplication domains and configured to:
receive a backup data object from a backup application, wherein the backup application created the backup data object, and a data object characteristic of the backup data object is not known to the computing device;
determine the data object characteristic of the backup data object based on metadata associated with the backup data object;
select a de-duplication domain from a plurality of de-duplication domains based at least in part on the data object characteristic of the backup data object; and
direct the backup data object to the selected de-duplication domain for de-duplication, wherein the selected de-duplication domain executes the de-duplication method that is associated with the selected de-duplication domain to de-duplicate redundant data among the backup data object, and one or more other backup data objects also directed to the selected de-duplication domain, based on the data object characteristic associated with the selected de-duplication domain.

### Claim 17 (depends on 16)

The system according to claim 16, wherein the controller is further configured to:
receive a data stream; and
identify the backup data object using metadata included in the data stream.

### Claim 18 (depends on 17)

The system according to claim 17, wherein the data stream is multiplexed.

### Claim 19 (depends on 17)

The system according to claim 17, wherein the controller is further configured to extract metadata included in the data stream with the backup data object.

### Claim 20 (depends on 19)

The system according to claim 19, wherein the controller is further configured to determine that the backup data object has the at least one characteristic associated with the de-duplication domain by comparing the extracted metadata associated with the backup data object to each set of data object characteristics associated with each de-duplication domain.

### Claim 21 (depends on 19)

The system according to claim 19, wherein the controller is further configured to extract at least one of a back-up policy name, a data source type, a data source name, a back-up application name, an operating system type, a data type, a back-up type, a filename, a directory structure and chronological information.

### Claim 22 (depends on 16)

The system according to claim 16, wherein the controller is further arranged to configure each of the plurality of de-duplication domains to use one of a plurality of de-duplication methods.

### Claim 23 (depends on 22)

The system according to claim 22, wherein the controller is further arranged to configure each of the plurality of de-duplication domains to use one de-duplication method selected from the group comprising hash-fingerprinting, pattern recognition and content aware de-duplication.

### Claim 24 (depends on 16)

The system according to claim 16, wherein the controller is further configured to associate each of the plurality of de-duplication domains with a set of one or more data object characteristics.

### Claim 25 (depends on 16)

The system according to claim 16, wherein the controller is further configured to:
cause, within the selected de-duplication domain, de-duplication of the backup data object; and
adjust the data object characteristic associated with at least one of the plurality of de-duplication domains based on a result of the act of de-duplicating.

### Claim 26 (depends on 25)

The system according to claim 25, wherein the controller is further configured to store data in a de-duplication domain database.

### Claim 27 (depends on 16)

The system according to claim 16, wherein the system is included in a grid computing environment.

### Claim 28 (depends on 16)

The system according to claim 16, wherein the controller is further configured to receive the backup data object, determine that the backup data object has the at least one characteristic associated with a de-duplication domain and direct the backup data object to the de-duplication domain while data is not being backed-up to the system.

### Claim 29 (depends on 16)

The system according to claim 16, wherein the controller is further configured to receive the backup data object, determine that the backup data object has the at least one characteristic associated with a de-duplication domain and direct the backup data object to the de-duplication domain while data is being backed-up to the system.
