Using a storage system to optimize and maintain the metadata associated with a plurality of small files

A computer program product, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions for:

• determining that a size associated with a first content file is less than a file threshold size, wherein metadata of the first content file is organized using a first file metadata structure, wherein the first file metadata structure comprises a first root node, a first plurality of intermediate nodes, and a first plurality of leaf nodes with corresponding references to data chunks of the first content file;
• in response to a determination that the size associated with the first content file is less than the file threshold size, updating a combined metadata structure at least in part by combining in the combined metadata structure the metadata of the first content file with metadata of a second content file included in a second file metadata structure, wherein the size associated with the second content file is less than the file threshold size, wherein the second file metadata structure comprises a second root node, a second plurality of intermediate nodes, and a second plurality of leaf nodes with corresponding references to data chunks of the second content file, wherein the combined metadata structure includes at least a third root node, the first plurality of leaf nodes and the second plurality of leaf nodes;
• and
• updating a snapshot tree to reference a first portion of the combined metadata structure corresponding to the first content file and to reference a second portion of the combined metadata structure corresponding to the second content file, wherein updating the snapshot tree to reference the first portion of the combined metadata structure corresponding to the first content file and to reference the second portion of the combined metadata structure corresponding to the second content file includes:
• using a combined metadata structure reservation node of the snapshot tree to reserve corresponding offset ranges of the combined metadata structure for the metadata of the first content file and the metadata of the second content file;
• and
• updating corresponding leaf nodes of the snapshot tree to indicate the corresponding offset ranges of the combined metadata structure for the metadata of the first content file and the metadata of the second content file, wherein the corresponding leaf nodes of the snapshot tree are updated to reference the third root node of the combined metadata structure.

A system, comprising:

• a processor configured to:
• determine that a size associated with a first content file is less than a file threshold size, wherein metadata of the first content file is organized using a first file metadata structure, wherein the first file metadata structure comprises a first root node, a first plurality of intermediate nodes, and a first plurality of leaf nodes with corresponding references to data chunks of the first content file;
• in response to determining that the size associated with the first content file is less than the file threshold size, update a combined metadata structure at least in part by combining in the combined metadata structure the metadata of the first content file with metadata of a second content file included in a second file metadata structure, wherein the size associated with the second content file is less than the file threshold size, wherein the second file metadata structure comprises a second root node, a second plurality of intermediate nodes, and a second plurality of leaf nodes with corresponding references to data chunks of the second content file, wherein the combined metadata structure includes at least a third root node, the first plurality of leaf nodes and the second plurality of leaf nodes;
• and
• update a snapshot tree to reference a first portion of the combined metadata structure corresponding to the first content file and to reference a second portion of the combined metadata structure corresponding to the second content file, wherein to update the snapshot tree to reference the first portion of the combined metadata structure corresponding to the first content file and to reference the second portion of the combined metadata structure corresponding to the second content file, the processor is configured to:
• use a combined metadata structure reservation node of the snapshot tree to reserve corresponding offset ranges of the combined metadata structure for the metadata of the first content file and the metadata of the second content file;
• and
• update corresponding leaf nodes of the snapshot tree to indicate the corresponding offset ranges of the combined metadata structure for the metadata of the first content file and the metadata of the second content file, wherein the corresponding leaf nodes of the snapshot tree are updated to reference the third root node of the combined metadata structure;
• and
• a memory coupled to the processor and configured to provide the processor with instructions.