System and method for identifying and mitigating redundancies in stored data
AssigneeFalconStor, Inc.
InventorWai Lam
Life of this patent
Estimating the term…
Abstract
In an example of an embodiment of the invention, a repeating pattern is identified within stored data comprising a plurality of data files, each data file comprising at least a header section and a data section stored in an unknown format. At least one occurrence of the repeating pattern is identified as a header section of a respective data file, and a data section of the respective data file is identified based, at least in part, on a location of the at least one occurrence of the repeating pattern. The identified data section of the respective data file is backed up. Systems are also disclosed.
Claims18
A method to manage data by one or more processing devices, the method comprising:
- identifying a repeating pattern in data stored in a storage system, by one of the one or more processing devices, the data comprising a plurality of pairs of header sections and data sections, wherein:
- a format of the header sections in the storage system are unknown to or cannot be processed by the processing device, and locations of the header sections are unknown to the processing device;
- and
- a format of the data sections are unknown to or cannot be processed by the processing device, and locations of the data sections are unknown to the processing device;
- associating the repeating pattern with the header sections;
- determining locations of the header sections in the plurality of pairs in the storage system based on the association;
- determining the locations of respective data sections in the plurality of pairs, in the storage system, based, at least in part, on the determined locations of respective header sections;
- and
- performing a selected data processing operation on the data sections in the determined locations by the one or more processing devices.
A system for performing data processing operations on stored data, comprising:
- storage configured to store data;
- and
- one or more processing devices configured to:
- identify a repeating pattern in data stored in a storage system the data comprising a plurality of pairs of header sections and data sections, wherein:
- a format of the header sections in the storage system are unknown to or cannot be processed by the processing device, and locations of the header sections are unknown to the processing device;
- and
- a format of the data sections are unknown to or cannot be processed by the processing device, and locations of the data sections are unknown to the processing device;
- determine locations of the header sections in the plurality of pairs, in the storage system, based on the association;
- determine the locations of respective data sections in the plurality of pairs, in the storage system, based, at least in part, on the determined locations of respective data sections;
- and
- perform a selected data processing operation on the data sections in the determined locations.
Dependencies are inferred from the claim text.