Methods and systems for efficiently storing data at a plurality of storage tiers using a transfer data structure

A non-transitory, machine readable medium having stored thereon instructions comprising machine executable code which when executed by a machine, causes the machine to:

• receive a request by a processor executing instructions out of a memory for a storage operating system to write a data block stored at a first storage tier to a second storage tier;
• use by the processor an object staging data structure to track an object that is in a process of being built to transfer a plurality of data blocks from the first storage tier to the second storage tier,
• wherein the object staging data structure is used to determine when the object is complete and available for transferring the plurality of data blocks from the first storage tier to the second storage tier;
• wherein the object staging data structure stores a unique identifier for the object, an indicator explicitly providing a status of the object indicating when the object is still in the process of being built and when the object is ready for transferring the plurality of data blocks, an object length indicating a length of the object and an offset value of a transfer log that stores physical addresses of the first storage tier where the plurality of data blocks are stored before the plurality of data blocks are transferred to the second tier;
• and
• wherein the transfer log is a temporary data structure that uses the object staging data structure to determine when the object is ready for transferring the plurality of data blocks to the second storage tier and the transfer log is used to process a read request that is received before any of the plurality of data blocks are transferred to the second storage tier;
• scan by the processor the object staging data structure to identify that the object is in the process of being built;
• update by the processor the transfer log with an address of the first storage tier where data regarding the data block is stored;
• increase by the processor the object length and the offset value at the object staging data structure for indicating metadata location of the data block within the object and an address for the data of the data block stored at the transfer log;
• generate by the processor the object identified by the object staging data structure indicating that the object is ready for transferring the data block;
• and
• transfer by the processor the data block and the plurality of data blocks of the object from the first storage tier to the second storage tier.

A system, comprising:

• a memory containing machine readable medium comprising machine executable code having stored thereon instructions;
• and a processor module coupled to the memory, the processor module configured to execute the machine executable code to:
• receive a request to write a data block stored at a first storage tier to a second storage tier;
• use an object staging data structure to track an object that is in a process of being built to transfer a plurality of data blocks from the first storage tier to the second storage tier,
• wherein the object staging data structure is used to determine when the object is complete and available for transferring the plurality of data blocks from the first storage tier to the second storage tier;
• wherein the object staging data structure stores a unique identifier for the object, an indicator explicitly providing a status of the object indicating when the object is still in the process of being built and when the object is ready for transferring the plurality of data blocks, an object length indicating a length of the object and an offset value of a transfer log that stores physical addresses of the first storage tier where for the plurality of data blocks are stored before the plurality of data blocks are transferred to the second tier;
• and
• wherein the transfer log is a temporary data structure that uses the object staging data structure to determine when the object is ready for transferring the plurality of data blocks to the second storage tier and the transfer log is used to process a read request that is received before any of the plurality of data blocks are transferred to the second storage tier;
• scan the object staging data structure to identify that the object is in the process of being built;
• update the transfer log with an address of the first storage tier where data regarding the data block is stored;
• increase the object length and the offset value at the object staging data structure for indicating metadata location of the data block within the object and an address for the data of the data block stored at the transfer log;
• generate the object identified by the object staging data structure indicating that the object is ready for transferring the data block;
• and
• transfer the data block and the plurality of data blocks of the object from the first storage tier to the second storage tier.