Performing an allreduce operation using shared memory

A compute node for performing an allreduce operation using shared memory, the compute node comprising a plurality of processing cores, computer memory operatively coupled to the plurality of processing cores, the computer memory having disposed within it computer program instructions capable of:

• receiving, by at least one of a plurality of processing cores on a compute node, an instruction to perform an allreduce operation, wherein the plurality of processing cores share the shared memory;
• establishing, by the core that received the instruction, a job status object that specifies a plurality of shared memory allreduce work units, the plurality of shared memory allreduce work units together perform the allreduce operation on the compute node, wherein the job status object is a data structure in shared memory that indicates, for each of the plurality of shared memory allreduce work units, at least one data element to be processed and a type of processing to be performed on the at least one data element;
• determining, by an available core of the plurality of processing cores on the compute node, a next shared memory allreduce work unit in the job status object to be processed;
• and
• performing, by that available core on the compute node, that next shared memory allreduce work unit.

A computer program product for performing an allreduce operation using shared memory, the computer program product disposed upon a computer-readable medium that is not a signal, the computer program product comprising computer program instructions capable of:

• receiving, by at least one of a plurality of processing cores on a compute node, an instruction to perform an allreduce operation, wherein the plurality of processing cores share the shared memory;
• establishing, by the core that received the instruction, a job status object that specifies a plurality of shared memory allreduce work units, the plurality of shared memory allreduce work units together perform the allreduce operation on the compute node, wherein the job status object is a data structure in shared memory that indicates, for each of the plurality of shared memory allreduce work units, at least one data element to be processed and a type of processing to be performed on the at least one data element;
• determining, by an available core of the plurality of processing cores on the compute node, a next shared memory allreduce work unit in the job status object to be processed;
• and
• performing, by that available core on the compute node, that next shared memory allreduce work unit.