Methods and systems for efficiently storing data

A non-transitory, machine readable medium having stored thereon instructions comprising machine executable code which when executed by a machine, causes the machine to:

• scan a first data structure by a processor executing instructions out of a memory for a storage operating system to determine whether any data chunk with one or more blocks of data of a first object having a plurality of data chunks stored at a first storage tier is referenced by the storage operating system;
• wherein the first data structure maintains a reference count for each data chunk of the first object indicating if the storage operating system is still using any data chunk and the first data structure is used to determine whether the first object is fragmented and ready for repackaging when a certain number of data chunks of the first object are referenced by the storage operating system or whether the first object is ready for deletion when no data chunk of the first object is being referenced by the storage operating system;
• when the storage operating system references the certain number of data chunks, the processor using an object staging data structure to identify a second object that is in the process of being built with enough space to repackage the certain number of data chunks from the first object to the second object and using the second object to store the certain number of data chunks;
• wherein the object staging data structure stores a unique identifier for identifying the second object, an indicator providing a status of the second object indicating that the object is being built, a length of the second object indicating a number of existing data blocks that are included in the second object and a transfer log file block number for providing a location within the transfer log where the existing data blocks of the second object are stored;
• and wherein the transfer log is a temporary data structure that uses the object staging data structure to determine when the second object is complete for storing the second object at the first storage tier with the certain number of data chunks of the first object;
• and
• update information regarding the second object at the transfer log with location information of the certain number of data chunks at the first storage tier.

A system, comprising:

• a memory containing machine readable medium comprising machine executable code having stored thereon instructions for a storage operating system;
• and a processor module coupled to the memory, the processor module configured to execute the machine executable code to:
• scan a first data structure to determine whether any data chunk with one or more blocks of data of a first object having a plurality of data chunks stored at a first storage tier is referenced by the storage operating system;
• wherein the first data structure maintains a reference count for each data chunk of the first object indicating if the storage operating system is still using any data chunk and the first data structure is used to determine whether the first object is fragmented and ready for repackaging when a certain number of data chunks of the first object are referenced by the storage operating system or whether the first object is ready for deletion when no data chunk of the first object is being referenced by the storage operating system;
• when the storage operating system references the certain number of data chunks, using an object staging data structure to identify a second object that is in the process of being built with enough space to repackage the certain number of data chunks from the first object to the second object and using the second object to store the certain number of data chunks;
• wherein the object staging data structure stores a unique identifier for identifying the second object, an indicator providing a status of the second object indicating that the object is being built, a length of the second object indicating a number of existing data blocks that are included in the second object and a transfer log file block number for providing a location within the transfer log where the existing data blocks of the second object are stored;
• and wherein the transfer log is a temporary data structure that uses the object staging data structure to determine when the second object is complete for storing the second object at the first storage tier with the certain number of data chunks of the first object;
• and
• update information regarding the second object at the transfer log with location information of the certain number of data chunks at the first storage tier.