Variable number and size of put operations for storage backup

A non-transitory machine readable medium having stored thereon instructions for performing a method comprising machine executable code which when executed by at least one machine, causes the machine to:

• receive a data stream for constructing a backup image for a data set;
• generate a plurality of data objects from the data stream, wherein the instructions to generate the plurality of data objects comprises instructions to,
• determine, for each multi-threaded put operation used to write a data object of the plurality of data objects to a cloud storage, a maximum number of threads into which the data object can be separated into a number of parts for simultaneous transmission to the cloud storage;
• and
• determine a number of multi-threaded put operations for writing the plurality of data objects to the cloud storage based, at least in part, on the maximum number of threads into which the data object can be separated;
• and
• store the plurality of data objects using the number of multi-threaded put operations into the cloud storage.

A computing device comprising:

• a memory containing machine readable medium comprising machine executable code having stored thereon instructions for performing a method of cluster management;
• and
• a processor coupled to the memory, the processor configured to execute the machine executable code to cause the processor to:
• receive a data stream for constructing a backup image for a data set;
• generate a plurality of data objects from the data stream, wherein the program code executable by the processor to cause the apparatus to generate the plurality of data objects comprises program code executable by the processor to cause the apparatus to,
• determine a stream size of the data stream;
• determine, for each multi-threaded put operation used to write a data object of the plurality of data objects to a cloud storage, a maximum number of threads into which the data object can be separated into a number of parts for simultaneous transmission to the cloud storage;
• determine a part size of each part of the number of parts;
• determine a number of multi-threaded put operations for writing the plurality of data objects to the cloud storage based, at least in part, on the stream size, the maximum number of threads into which the data object can be separated, and the part size;
• and
• store the plurality of data objects using the number of multi-threaded put operations into the cloud storage.