Back to results
US 2010/0161685 A1Published application
Methods and apparatus for content-aware data partitioning
AssigneeSumooh Inc.
InventorsSanjay JAIN, Puneesh CHAUDHRY
Life of this patent
Estimating the term…
Abstract
The systems and methods partition digital data units in a content aware fashion without relying on any ancestry information, which enables one to find duplicate chunks in unrelated units of digital data even across millions of documents spread across thousands of computer systems.
Claims32
A computer-implemented method for partitioning and storing digital data comprising:
- determining a format of the digital data;
- identifying a source logical object within the digital data, wherein the identification of the source logical object is accomplished at least in part by applying knowledge about the format of the digital data;
- based on the determined format, performing one or more of the following operations to create a resulting object:
- removing position-dependent data from the source logical object,
- removing instance-dependent data from the source logical object,
- removing one or more format-specific headers or footers from the source logical object, and
- removing format-specific transformations from the source logical object;
- determining whether the resulting object has already been stored;
- and
- storing the resulting object if the resulting object has not already been stored.
7 more elements
A system comprising:
- a memory capable of storing data;
- and
- a processor configured for:
- determining a format of digital data;
- identifying a source logical object within the digital data, wherein the identification of the source logical object is accomplished at least in part by applying knowledge about the format of the digital data;
- based on the determined format, performing one or more of the following operations to create a resulting object:
- removing position-dependent data from the source logical object,
- removing instance-dependent data from the source logical object,
- removing one or more format-specific headers or footers from the source logical object, and
- removing format-specific transformations from the source logical object,
- determining whether the resulting object has already been stored in the memory, and
- storing the resulting object in the memory if the resulting object has not already been stored.
9 more elements
A computer-implemented method for partitioning digital data comprising:
- determining a format of the digital data;
- identifying a source logical object within the digital data, wherein the identification of the source logical object is accomplished at least in part by applying knowledge about the format of the digital data;
- determining whether the resulting object has already been stored;
- storing the resulting object if the resulting object has not already been stored;
- storing additional data, wherein the additional data comprises one or more of information indicative of the original position of the logical object within the digital data, position-dependent information, instance-dependent information, format-specific headers or footers, and information indicative of format-specific transformations;
- associating the additional data with the resulting object;
- and
- using the stored resulting object and the associated additional data to recreate the digital data.
5 more elements
A system comprising:
- a memory capable of storing data;
- and
- a processor configured for:
- determining a format of digital data,
- identifying a source logical object within the digital data, wherein the identification of the source logical object is accomplished at least in part by applying knowledge about the format of the digital data,
- determining whether the resulting object has already been stored,
- storing the resulting object if the resulting object has not already been stored,
- storing additional data, wherein the additional data comprises one or more of information indicative of the original position of the logical object within the digital data, position-dependent information, instance-dependent information associating the additional data with the resulting object, and
- using the stored resulting object and the associated additional data to recreate the digital data.
6 more elements
Dependencies are inferred from the claim text.