Dynamic assignment of transfers of blocks of data

A computer-implemented method comprising:

• transmitting, from a distribution thread, a command via a network to a data storage cluster for each data transfer thread of a multitude of data transfer threads executed within the data storage cluster to request assignment of a data transfer from the distribution thread at a network address specified in the command;
• awaiting receipt of a request for assignment from a data transfer thread of the multitude of data transfer threads via the network;
• comparing a quantity of threads of the multitude of data transfer threads to a quantity of threads of a multitude of computation threads executed within a data processing cluster;
• and
• assigning to the data transfer thread an exchange of a block of data with a single computation thread of the multitude of computation in response to receiving the request for assignment and in response to the multitude of data transfer threads comprising a greater quantity of threads than the multitude of computation threads, the single computation thread to perform one or more computations on the block of data, or
• assigning to the data transfer thread exchanges of multiple blocks of data with multiple computation threads of the multitude of computation threads in response to receiving the request for assignment and in response to the multitude of data transfer threads comprising a lesser quantity of threads than the multitude of computation threads, wherein the multiple blocks of data comprises the block of data, the multiple computation threads to each perform one or more computations on one or more of the multiple blocks of data.

An apparatus comprising:

• a processor component;
• a command component for execution by the processor component in a distribution thread to transmit a command via a network to a data storage cluster for each data transfer thread of a multitude of data transfer threads executed within the data storage cluster to request assignment of a data transfer from an assignment component at a network address on the network specified in the command;
• and
• the assignment component for execution by the processor component in the distribution thread to:
• await receipt of a request for assignment from a data transfer thread of the multitude of data transfer threads via the network;
• compare a quantity of threads of the multitude of data transfer threads to a quantity of threads of a multitude of computation threads executed within a data processing cluster;
• assign to the data transfer thread an exchange of a block of data with a single computation thread of the multitude of computation in response to receipt of the request for assignment and in response to the multitude of data transfer threads comprising a greater quantity of threads than the multitude of computation threads, the single computation thread to perform one or more computations on the block of data;
• and
• assign to the data transfer thread exchanges of multiple blocks of data with multiple computation threads of the multitude of computation threads in response to receipt of the request for assignment and in response to the multitude of data transfer threads comprising a lesser quantity of threads than the multitude of computation threads, wherein the multiple blocks of data comprises the block of data, the multiple computation threads to each perform one or more computations on one or more of the multiple blocks of data.