# US 2014/0195493 A1 · Packing deduplicated data in a self-contained deduplicated repository > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2014/0195493 A1 - Document ID: US20140195493A1 - Status: Published application - Kind: A1 - Application number: 13/738,002 - Filed: 2013-01-10 - Published: 2014-07-10 - Earliest claimed: 2013-01-10 - Assignees: International Business Machines Corporation - Inventors: Shay H. AKIRAV; Michael HIRSCH; Ofer LENEMAN - Web page: https://patentgrep.com/patent/US20140195493A1 ## CPC - G06F 16/1748: {De-duplication implemented within the file system, e.g. based on file segments (de-duplication techniques in storage systems for the management of data blocks G06F3/0641)} - G06F 3/0608: {Saving storage space on storage systems} - G06F 3/0641: {De-duplication techniques} - G06F 3/067: {Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Deduplicated data is packed in a self-contained deduplicated repository having unique data blocks with each being referenced by a globally unique identifier (GUID). The self-contained deduplicated repository has information regarding both deduplicated data files and the unique data blocks of each of the deduplicated data files and a master GUID list containing a location of each of the unique data blocks. ## Claims ### Claim at position 1 1-12. (canceled) ### Claim 13 A system for storing deduplicated data in a computing environment, the system comprising: a processor device operable in the computing storage environment, wherein the processor device: packs deduplicated data in a self-contained deduplicated repository having a plurality of unique data blocks with each being referenced by a globally unique identifier (GUID), and the self-contained deduplicated repository also having information regarding both deduplicated data files and the plurality of unique data blocks of each of the deduplicated data files and a master GUID list containing a location of each of the plurality of unique data blocks. ### Claim 14 (depends on 13) The system of claim 13, wherein the processor device loading the self-contained deduplicated repository into a target repository. ### Claim 15 (depends on 13) The system of claim 13, wherein the processor device creates the self-contained deduplicated repository from a list of deduplicated data files, wherein the self-contained deduplicated repository is isolated and contains at least one of a table of contents (TOC) of the deduplicated data files, metadata of the deduplicated data files, and one unique copy of each of the plurality of unique data blocks. ### Claim 16 (depends on 15) The system of claim 15, wherein the processor device examines the TOC of each of the deduplicated data files in the list of deduplicated data files one after another, wherein the master GUID list is maintained, and the TOC is an ordered list of the GUID that make up a file. ### Claim 17 (depends on 15) The system of claim 15, wherein the processor device adds to the master GUID list each one of the plurality of unique data blocks required by any one of the deduplicated data files in the list of deduplicated data files. ### Claim 18 (depends on 15) The system of claim 15, wherein the processor device commits to the self-contained deduplicated repository the TOC and metadata of each of the deduplicated data files in the list of deduplicated data files, the master GUID list, and the plurality of unique data blocks required by any one of the deduplicated data files in the list of deduplicated data files. ### Claim 19 (depends on 15) The system of claim 15, wherein the processor device performs one of: ordering the plurality of unique data blocks in the self-contained deduplicated repository by the GUID, and mapping a position of the plurality of unique data blocks in the self-contained deduplicated repository. ### Claim 20 (depends on 15) The system of claim 15, wherein the processor device performs each of: inputting into the self-contained deduplicated repository the list of deduplicated data files, determining if each one of the deduplicated data files in the list of deduplicated data files are processed, appending metadata and the TOC of the one of the deduplicated data files to the self-contained deduplicated repository if one of the deduplicated data files is not processed, determining if each of the GUID in the one of the deduplicated data files is processed, determining if the GUID is located in the master GUID list, adding the GUID to the master GUID list, writing the master GUID list to the self-contained deduplicated repository, and writing the data of each of the GUID to the self-contained deduplicated repository. ### Claim 21 (depends on 15) The system of claim 15, wherein the processor device retrieves the deduplicated data file from the self-contained deduplicated repository. ### Claim 22 (depends on 21) The system of claim 21, wherein the processor device performs one of: scanning the list of deduplicated data files at the beginning of the self-contained deduplicated repository until locating one of the deduplicated data files to be retrieved, retrieving metadata of the one of the deduplicated data files and the TOC from the self-contained deduplicated repository, skipping to the master GUID list and retrieving the master GUID list from the self-contained deduplicated repository, if each of the GUID in the one of the deduplicated data files is not processed, performing each of: finding a position of a first GUID in the one of the deduplicated data files in the master GUID list, and adding a tuple to the master GUID list that is retrieved, and if each of the GUID in the one of the deduplicated data files is processed, performing each of: sorting the master GUID list that is retrieved by position, determining if all tuples in the master GUID list have been processed, deriving a position of the GUID of a first tuple in the self-contained deduplicated repository from the position in the master GUID list that is retrieved, forwarding to the position of the GUID of the first tuple in the self-contained deduplicated repository and retrieving data of the GUID, and writing the data of the GUID of the first tuple to each position in the one of the deduplicated data according to appearance in consecutive tuples in the master GUID list that is sorted. ### Claim 23 (depends on 15) The system of claim 15, wherein the processor device performs one of: restoring the deduplicated data file from the self-contained deduplicated repository by reading a table of contents (TOC) of the deduplicated data file and reading each one of the plurality of unique data blocks required by the deduplicated data file using the GUID listed in the TOC, and merging all deduplicated data files in the self-contained deduplicated repository into a target repository by reading the TOC and deduplication metadata, wherein the TOC and deduplication metadata are stored, and any of the GUID needed by the TOC that do not exist in the target repository are read from the self-contained deduplicated repository and stored in the target repository. ### Claim 24 (depends on 13) The system of claim 13, wherein the processor device allows each deduplication system that draws the GUID for the deduplication system from a similar GUID allocation sphere to share each one of a plurality of self-contained deduplicated repositories to be shared for, extract files from each one of the plurality of self-contained deduplicated repositories, and append to each one of the plurality of self-contained deduplicated repositories. ### Claim 25 A computer program product storing deduplicated data by a processor device, the computer program product comprising a computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions comprising: a first executable portion that packs deduplicated data in a self-contained deduplicated repository having a plurality of unique data blocks with each being referenced by a globally unique identifier (GUID), and the self-contained deduplicated repository also having information regarding both deduplicated data files and the plurality of unique data blocks of each of the deduplicated data files and a master GUID list containing a location of each of the plurality of unique data blocks. ### Claim 26 (depends on 25) The computer program product of claim 25, further including a second executable portion that loads the self-contained deduplicated repository into a target repository. ### Claim 27 (depends on 25) The computer program product of claim 25, further including a third executable portion that creates the self-contained deduplicated repository from a list of deduplicated data files, wherein the self-contained deduplicated repository is isolated and contains at least one of a table of contents (TOC) of the deduplicated data files, metadata of the deduplicated data files, and one unique copy of each of the plurality of unique data blocks. ### Claim 28 (depends on 27) The computer program product of claim 27, further including a fourth executable portion that examines the TOC of each of the deduplicated data files in the list of deduplicated data files one after another, wherein the master GUID list is maintained, and the TOC is an ordered list of the GUID that make up a file. ### Claim 29 (depends on 27) The computer program product of claim 27, further including a fourth executable portion that adds to the master GUID list each one of the plurality of unique data blocks required by any one of the deduplicated data files in the list of deduplicated data files. ### Claim 30 (depends on 27) The computer program product of claim 27, further including a fourth executable portion that commits to the self-contained deduplicated repository the TOC and metadata of each of the deduplicated data files in the list of deduplicated data files, the master GUID list, and the plurality of unique data blocks required by any one of the deduplicated data files in the list of deduplicated data files. ### Claim 31 (depends on 27) The computer program product of claim 27, further including a fourth executable portion that performs one of: ordering the plurality of unique data blocks in the self-contained deduplicated repository by the GUID, and mapping a position of the plurality of unique data blocks in the self-contained deduplicated repository. ### Claim 32 (depends on 27) The computer program product of claim 27, further including a fourth executable portion that performs each of: inputting into the self-contained deduplicated repository the list of deduplicated data files, determining if each one of the deduplicated data files in the list of deduplicated data files are processed, appending metadata and the TOC of the one of the deduplicated data files to the self-contained deduplicated repository if one of the deduplicated data files is not processed, determining if each of the GUID in the one of the deduplicated data files is processed, determining if the GUID is located in the master GUID list, adding the GUID to the master GUID list, writing the master GUID list to the self-contained deduplicated repository, and writing the data of each of the GUID to the self-contained deduplicated repository. ### Claim 33 (depends on 27) The computer program product of claim 27, further including a fourth executable portion that retrieves the deduplicated data file from the self-contained deduplicated repository. ### Claim 34 (depends on 33) The computer program product of claim 33, further including a fifth executable portion that retrieves the deduplicated data file from the self-contained deduplicated repository by performing one of: scanning the list of deduplicated data files at the beginning of the self-contained deduplicated repository until locating one of the deduplicated data files to be retrieved, retrieving metadata of the one of the deduplicated data files and the TOC from the self-contained deduplicated repository, skipping to the master GUID list and retrieving the master GUID list from the self-contained deduplicated repository, if each of the GUID in the one of the deduplicated data files is not processed, performing each of: finding a position of a first GUID in the one of the deduplicated data files in the master GUID list, and adding a tuple to the master GUID list that is retrieved, and if each of the GUID in the one of the deduplicated data files is processed, performing each of: sorting the master GUID list that is retrieved by position, determining if all tuples in the master GUID list have been processed, deriving a position of the GUID of a first tuple in the self-contained deduplicated repository from the position in the master GUID list that is retrieved, forwarding to the position of the GUID of the first tuple in the self-contained deduplicated repository and retrieving data of the GUID, and writing the data of the GUID of the first tuple to each position in the one of the deduplicated data according to appearance in consecutive tuples in the master GUID list that is sorted. ### Claim 35 (depends on 27) The computer program product of claim 27, further including a fourth executable portion that performs one of: restoring the deduplicated data file from the self-contained deduplicated repository by reading a table of contents (TOC) of the deduplicated data file and reading each one of the plurality of unique data blocks required by the deduplicated data file using the GUID listed in the TOC, and merging all deduplicated data files in the self-contained deduplicated repository into a target repository by reading the TOC and deduplication metadata, wherein the TOC and deduplication metadata are stored, and any of the GUID needed by the TOC that do not exist in the target repository are read from the self-contained deduplicated repository and stored in the target repository. ### Claim 36 (depends on 25) The computer program product of claim 25, further including a third executable portion that allows each deduplication system that draws the GUID for the deduplication system from a similar GUID allocation sphere to share each one of a plurality of self-contained deduplicated repositories to be shared for, extract files from each one of the plurality of self-contained deduplicated repositories, and append to each one of the plurality of self-contained deduplicated repositories.