# US 2017/0286002 A1 · Region-integrated data deduplication > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2017/0286002 A1 - Document ID: US20170286002A1 - Status: Published application - Kind: A1 - Application number: 15/084,410 - Filed: 2016-03-29 - Published: 2017-10-05 - Earliest claimed: 2016-03-29 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: Jonathan Amit; Aviv Caro; David D. Chambliss; Joseph S. Glider; Chaim Koifman; Yosef Shatsky - Web page: https://patentgrep.com/patent/US20170286002A1 ## CPC - G06F 3/0641: {De-duplication techniques} - G06F 3/0608: {Saving storage space on storage systems} - G06F 3/0673: {Single storage device} - G06F 3/0683: {Plurality of storage devices} - G06F 16/1748: {De-duplication implemented within the file system, e.g. based on file segments (de-duplication techniques in storage systems for the management of data blocks G06F3/0641)} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract In one embodiment, a computer program product is configured for performing deduplication in conjunction with random read and write operations across a namespace divided into a plurality of disjoint regions. The computer program product includes a computer readable storage medium having program instructions embodied therewith, where the computer readable storage medium is not a transitory signal per se. The program instructions are executable by a computer to cause the computer to perform a method including: maintaining a metadata structure for each of the plurality of disjoint regions via the respective region manager(s) of the plurality of disjoint regions, each metadata structure comprising metadata indicating a physical storage location of one or more data chunks associated with the respective region; and performing, by the computer, a deduplicated write operation of a first data chunk in a first region of the plurality of disjoint regions. ## Claims ### Claim 1 A computer program product for performing deduplication in conjunction with random read and write operations across a namespace divided into a plurality of disjoint regions, each region of the namespace being managed by a region manager, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, wherein the computer readable storage medium is not a transitory signal per se, the program instructions executable by a computer to cause the computer to perform a method comprising: maintaining, by the computer, a metadata structure for each of the plurality of disjoint regions via the respective region manager(s) of the one or more of the plurality of disjoint regions, each metadata structure comprising metadata indicating a physical storage location of one or more data chunks associated with the respective region; and performing, by the computer, a deduplicated write operation of a first data chunk in a first region of the plurality of disjoint regions, wherein the deduplicated write operation comprises: consulting, by a first region manager of the first region, a duplicate chunk finder in response to the first region manager receiving a write request corresponding to the first data chunk; determining, using at least the duplicate chunk finder, whether one or more of the plurality of disjoint regions other than the first region comprises a second data chunk identical in content to the first data chunk; and in response to determining one or more of the plurality of disjoint regions other than the first region comprises the second data chunk, establishing by the first region manager a reference from the first data chunk to the second data chunk. ### Claim 2 (depends on 1) The computer program product as recited in claim 1, wherein each region manager is configured to service one or more of random read and/or random write operations directed to one or more of the plurality of disjoint regions via the respective region manager(s) of the one or more of the plurality of disjoint regions. ### Claim 3 (depends on 1) The computer program product as recited in claim 1, wherein establishing the reference from the first data chunk to the second data chunk involves the region managers of the regions hosting the first data chunk and the second data chunk. ### Claim 4 (depends on 1) The computer program product as recited in claim 1, wherein determining whether the one or more of the plurality of disjoint regions other than the first region comprise the second data chunk is performed further using the respective region managers of the one or more of the plurality of disjoint regions. ### Claim 5 (depends on 1) The computer program product as recited in claim 1, comprising program instructions executable by the computer to cause the computer to receive a confirmation from at least one region manager of the one or more of the plurality of disjoint regions other than the first region that comprise the second data chunk. ### Claim 6 (depends on 1) The computer program product as recited in claim 1, wherein the metadata structure for each region comprises metadata indicating a number of regions having a reference to the one or more data chunks associated with the respective region. ### Claim 7 (depends on 1) The computer program product as recited in claim 1, wherein the metadata structure for each region comprises metadata indicating a number of references to the one or more data chunks associated with the respective region. ### Claim 8 (depends on 1) The computer program product as recited in claim 1, wherein the metadata structure for each region comprises metadata indicating a unique identifier of the one or more data chunks associated with the respective region. ### Claim 9 (depends on 8) The computer program product as recited in claim 8, wherein each unique identifier comprises a 20-byte hash based on content of the corresponding data chunk. ### Claim 10 (depends on 1) The computer program product as recited in claim 1, comprising a virtual namespace map comprising: metadata indicating a unique key for each data chunk stored in the namespace; and metadata indicating a region to which each data chunk stored in the namespace belongs. ### Claim 11 (depends on 1) The computer program product as recited in claim 1, comprising program instructions configured to cause the computer to perform a deduplicated read operation directed to the namespace. ### Claim 12 (depends on 11) The computer program product as recited in claim 11, wherein the deduplicated read operation comprises, in response to the first region manager receiving a read request corresponding to the deduplicated read operation: determining, by the first region manager, whether a reference to the second data chunk exists in the metadata structure of the first region; and in response to determining the reference to the second data chunk exists in the metadata structure of the first region, providing access to the second data chunk by a second region manager managing the region hosting the second data chunk. ### Claim 13 (depends on 11) The computer program product as recited in claim 11, wherein the deduplicated read operation comprises, in response to the first region manager receiving a read request corresponding to the deduplicated read operation: determining, by the first region manager, whether a reference to the second data chunk exists in the metadata structure of the first region; and in response to determining the reference to the second data chunk exists in the metadata structure of the first region, reading by the first region manager the second data chunk. ### Claim 14 (depends on 1) The computer program product as recited in claim 1, wherein establishing the reference to the second data chunk is performed only in response to a first instance of the determining one or more of the plurality of disjoint regions other than the first region comprises the second data chunk. ### Claim 15 (depends on 1) The computer program product as recited in claim 1, comprising program instructions configured to cause the computer to perform a non-deduplicated write of the first data chunk to the first region in response to determining none of the plurality of disjoint regions other than the first region comprises the second data chunk. ### Claim 16 (depends on 15) The computer program product as recited in claim 15, comprising creating a metadata record in the metadata structure of the first region in response to performing the non-deduplicated write of the first data chunk, the metadata record comprising at least a unique identifier corresponding to the first data chunk and a physical storage location corresponding to the first data chunk. ### Claim 17 (depends on 1) The computer program product as recited in claim 1, wherein each data chunk is characterized by a size in a region from about 4 KB to about 16 KB. ### Claim 18 (depends on 1) The computer program product as recited in claim 1, wherein the namespace is implemented on a primary storage system. ### Claim 19 A computer-implemented method for performing deduplication in conjunction with random read and write operations across a namespace divided into a plurality of disjoint regions, each region of the namespace being managed by a region manager, the method comprising: maintaining a metadata structure for each of the plurality of disjoint regions via the respective region manager(s) of the one or more of the plurality of disjoint regions, each metadata structure comprising metadata indicating a physical storage location of one or more data chunks associated with the respective region; and performing a deduplicated write operation of a first data chunk in a first region of the plurality of disjoint regions, wherein the deduplicated write operation comprises: consulting, by a first region manager of the first region, a duplicate chunk finder in response to the first region manager receiving a write request corresponding to the first data chunk; determining, using at least the duplicate chunk finder, whether one or more of the plurality of disjoint regions other than the first region comprises a second data chunk identical in content to the first data chunk; and in response to determining one or more of the plurality of disjoint regions other than the first region comprises the second data chunk, establishing by the first region manager a reference from the first data chunk to the second data chunk. ### Claim 20 A deduplicating storage system configured to perform random read and write operations across a namespace, the system comprising: a processor and logic integrated with and/or executable by the processor, the logic being configured to cause the processor to perform a method comprising: maintaining a metadata structure for each of the plurality of disjoint regions via the respective region manager(s) of the one or more of the plurality of disjoint regions, each metadata structure comprising metadata indicating a physical storage location of one or more data chunks associated with the respective region; and performing a deduplicated write operation of a first data chunk in a first region of the plurality of disjoint regions, wherein the deduplicated write operation comprises: consulting, by a first region manager of the first region, a duplicate chunk finder in response to the first region manager receiving a write request corresponding to the first data chunk; determining, using at least the duplicate chunk finder, whether one or more of the plurality of disjoint regions other than the first region comprises a second data chunk identical in content to the first data chunk; and in response to determining one or more of the plurality of disjoint regions other than the first region comprises the second data chunk, establishing by the first region manager a reference from the first data chunk to the second data chunk.