Optmizing metadata management in data deduplication

A method of processing data comprising:

• receiving a candidate block including a plurality of uniformly-sized sub-blocks, wherein a tag is stored at a first location in the candidate block;
• performing data deduplication processing of the candidate block, wherein the data deduplication processing excludes content stored at the first location, and wherein a first offset to a second offset of the candidate block corresponds to the first location;
• determining whether at least one sub-block of the candidate block has been deduplicated by the data deduplication processing;
• and
• responsive to determining that at least one sub-block of the candidate block has been deduplicated, storing the candidate block as a deduplicated data block having at least one sub-block matching an existing target sub-block, wherein a tag descriptor describing the tag is stored and associated with the candidate block, and wherein the method includes performing first processing that determines whether the candidate block has the tag, said first processing including:
• scanning the candidate block from a right-most sub-block and determining the second offset at which a difference is detected between the candidate block and a target block;
• scanning the candidate block from a left-most sub-block and determining the first offset at which a difference is detected between the candidate block and the target block;
• determining whether a distance between the first offset and the second offset is less than a maximum tag size;
• and
• responsive to determining the distance is less than the maximum tag size, determining that the candidate block has the tag with an associated tag location from the first offset to the second offset in the candidate block.

A system comprising:

• at least one processor;
• and
• a memory comprising code stored thereon that, when executed, performs a method of processing data comprising:
• receiving a candidate block including a plurality of uniformly-sized sub-blocks, wherein a tag is stored at a first location in the candidate block;
• performing data deduplication processing of the candidate block, wherein the data deduplication processing excludes content stored at the first location, and wherein a first offset to a second offset of the candidate block corresponds to the first location, wherein a tag descriptor includes tag content and includes location information identifying the first location in the candidate block, and wherein the tag content is stored at the first location in the candidate block;
• determining whether at least one sub-block of the candidate block has been deduplicated by the data deduplication processing;
• and
• responsive to determining that at least one sub-block of the candidate block has been deduplicated, storing the candidate block as a deduplicated data block having at least one sub-block matching an existing target sub-block, wherein the tag descriptor describing the tag is stored and associated with the candidate block.

A non-transitory computer readable medium comprising code stored thereon that, when executed, performs a method of processing data comprising:

• receiving a candidate block including a plurality of uniformly-sized sub-blocks, wherein a tag is stored at a first location in the candidate block;
• performing data deduplication processing of the candidate block, wherein the data deduplication processing excludes content stored at the first location, and wherein a first offset to a second offset of the candidate block corresponds to the first location, wherein a tag descriptor includes tag content and includes location information identifying the first location in the candidate block, and wherein the tag content is stored at the first location in the candidate block;
• determining whether at least one sub-block of the candidate block has been deduplicated by the data deduplication processing;
• and
• responsive to determining that at least one sub-block of the candidate block has been deduplicated, storing the candidate block as a deduplicated data block having at least one sub-block matching an existing target sub-block, wherein the tag descriptor describing the tag is stored and associated with the candidate block.

A system comprising:

• at least one processor;
• and
• a memory comprising code stored thereon that, when executed, performs a method of processing data comprising:
• receiving a candidate block including a plurality of uniformly-sized sub-blocks, wherein a tag is stored at a first location in the candidate block;
• performing data deduplication processing of the candidate block, wherein the data deduplication processing excludes content stored at the first location, and wherein a first offset to a second offset of the candidate block corresponds to the first location;
• determining whether at least one sub-block of the candidate block has been deduplicated by the data deduplication processing;
• and
• responsive to determining that at least one sub-block of the candidate block has been deduplicated, storing the candidate block as a deduplicated data block having at least one sub-block matching an existing target sub-block, wherein a tag descriptor describing the tag is stored and associated with the candidate block, and wherein the method includes performing first processing that determines whether the candidate block has the tag, said first processing including:
• scanning the candidate block from a right-most sub-block and determining the second offset at which a difference is detected between the candidate block and a target block;
• scanning the candidate block from a left-most sub-block and determining the first offset at which a difference is detected between the candidate block and the target block;
• determining whether a distance between the first offset and the second offset is less than a maximum tag size;
• and
• responsive to determining the distance is less than the maximum tag size, determining that the candidate block has the tag with an associated tag location from the first offset to the second offset in the candidate block.

A non-transitory computer readable medium comprising code stored thereon that, when executed, performs a method of processing data comprising:

• receiving a candidate block including a plurality of uniformly-sized sub-blocks, wherein a tag is stored at a first location in the candidate block;
• performing data deduplication processing of the candidate block, wherein the data deduplication processing excludes content stored at the first location, and wherein a first offset to a second offset of the candidate block corresponds to the first location;
• determining whether at least one sub-block of the candidate block has been deduplicated by the data deduplication processing;
• and
• responsive to determining that at least one sub-block of the candidate block has been deduplicated, storing the candidate block as a deduplicated data block having at least one sub-block matching an existing target sub-block, wherein a tag descriptor describing the tag is stored and associated with the candidate block, and wherein the method includes performing first processing that determines whether the candidate block has the tag, said first processing including:
• scanning the candidate block from a right-most sub-block and determining the second offset at which a difference is detected between the candidate block and a target block;
• scanning the candidate block from a left-most sub-block and determining the first offset at which a difference is detected between the candidate block and the target block;
• determining whether a distance between the first offset and the second offset is less than a maximum tag size;
• and
• responsive to determining the distance is less than the maximum tag size, determining that the candidate block has the tag with an associated tag location from the first offset to the second offset in the candidate block.