Management of data staging for writing to object based storage

A computing device comprising:

• a memory containing a machine-readable medium having instructions stored thereon comprising machine executable code;
• and
• a processor coupled to the memory, the processor configured to execute the machine executable code to:
• receive a request to write data to an object based storage;
• determine that the data is to be staged within a transfer data structure before being written to the object based storage,
• wherein the transfer data structure is in a first storage tier and the object based storage is in a second storage tier;
• determine that an amount of storage space used in the transfer data structure exceeds a start throttle threshold;
• send, in response to a determination the amount of storage space used exceeds the start throttle threshold, the request into a queue selected from a set of throttling queues;
• process the request in the queue based on a dynamic throttle rate;
• and
• adjust a drain throttle rate at which the data is transferred out of the transfer data structure based on whether the data to be written is first stored in a physical storage in the first storage tier separate from the transfer data structure.

A non-transitory machine-readable medium having stored thereon instructions for performing a method comprising machine-executable code which, when executed by at least one machine, causes the at least one machine to:

• receive a request to stage data in a transfer data structure in a first storage tier before the data is written to object based storage in a second storage tier;
• determine that an amount of storage space used in the transfer data structure exceeds a start throttle threshold;
• send, in response to a determination the amount of storage space used exceeds the start throttle threshold, the request into a queue selected from a set of throttling queues;
• identify a dynamic throttle rate based on a dynamic drain rate computed for the transfer data structure;
• process the request in the queue based on the dynamic throttle rate, a priority type of the queue, and a first in, first out (FIFO) order;
• and
• adjust a drain throttle rate at which the data is transferred out of the transfer data structure based on whether the data to be written is first stored in a physical storage in the first storage tier separate from the transfer data structure.