# US 8,752,051 B2 · Performing an allreduce operation using shared memory > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 8,752,051 B2 - Document ID: US8752051B2 - Status: Granted - Kind: B2 - Application number: 13/427,057 - Filed: 2012-03-22 - Published: 2012-07-12 - Granted: 2014-06-10 - Earliest claimed: 2007-05-29 - Assignees: International Business Machines Corporation - Applicants: Charles J. Archer; Gabor Dozsa; Joseph D. Ratterman; Brian E. Smith - Inventors: Charles J. Archer; Gabor Dozsa; Joseph D. Ratterman; Brian E. Smith - Web page: https://patentgrep.com/patent/US8752051B2 ## CPC - G06F 9/4843: {by program, e.g. task dispatcher, supervisor, operating system} - G06F 9/546: {Message passing systems or structures, e.g. queues} - G06F 9/52: Program synchronisation; Mutual exclusion, e.g. by means of semaphores - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Methods, apparatus, and products are disclosed for performing an allreduce operation using shared memory that include: receiving, by at least one of a plurality of processing cores on a compute node, an instruction to perform an allreduce operation; establishing, by the core that received the instruction, a job status object for specifying a plurality of shared memory allreduce work units, the plurality of shared memory allreduce work units together performing the allreduce operation on the compute node; determining, by an available core on the compute node, a next shared memory allreduce work unit in the job status object; and performing, by that available core on the compute node, that next shared memory allreduce work unit. ## Claims ### Claim 1 A method for performing an allreduce operation using shared memory, the method comprising: receiving, by at least one of a plurality of processing cores on a compute node, an instruction to perform an allreduce operation, wherein the plurality of processing cores share the shared memory; establishing, by the core that received the instruction, a job status object that specifies a plurality of shared memory allreduce work units, the plurality of shared memory allreduce work units together perform the allreduce operation on the compute node, wherein the job status object is a data structure in shared memory that indicates, for each of the plurality of shared memory allreduce work units, at least one data element to be processed and a type of processing to be performed on the at least one data element; determining, by an available core of the plurality of processing cores on the compute node, a next shared memory allreduce work unit in the job status object to be processed; and performing, by that available core on the compute node, that next shared memory allreduce work unit. ### Claim 2 (depends on 1) The method of claim 1 wherein establishing, by the core that received the instruction, a job status object for specifying a plurality of shared memory allreduce work units further comprises assigning a plurality of threads for executing the shared memory allreduce work units. ### Claim 3 (depends on 1) The method of claim 1 further comprising: copying, by each available core on the compute node into separate shared memory buffers, an array to be reduced by the allreduce operation; and updating, by each available core on the compute node, the job status object with a descriptor of the shared memory buffer into which the available core copied the array. ### Claim 4 (depends on 1) The method of claim 1 further comprising: providing, by each available core to each other available core, a read-only window into a local memory buffer containing an array to be reduced by the allreduce operation; and updating, by each available core on the compute node, the job status object with a descriptor of the buffer into which the read-only window is provided. ### Claim 5 (depends on 1) The method of claim 1 wherein performing, by that available core on the compute node, that next shared memory allreduce work unit further comprises performing a reduction operation on elements of arrays to be reduced. ### Claim 6 (depends on 1) The method of claim 1 wherein: the compute node is connected to a plurality of other compute nodes through a data communications network; and performing, by that available core on the compute node, that next shared memory allreduce work unit further comprises transmitting, by that available core, local reduction results to one or more of the other compute nodes through the network. ### Claim 7 (depends on 1) The method of claim 1 wherein: the compute node is connected to a plurality of other compute nodes through a data communications network; performing, by that available core on the compute node, that next shared memory allreduce work unit further comprises: receiving global reduction results through the network, and storing the global reduction results into shared memory; and the method further comprises copying, by each available core on the compute node, the global reduction results from the shared memory to local memory of the available core. ### Claim 8 A compute node for performing an allreduce operation using shared memory, the compute node comprising a plurality of processing cores, computer memory operatively coupled to the plurality of processing cores, the computer memory having disposed within it computer program instructions capable of: receiving, by at least one of a plurality of processing cores on a compute node, an instruction to perform an allreduce operation, wherein the plurality of processing cores share the shared memory; establishing, by the core that received the instruction, a job status object that specifies a plurality of shared memory allreduce work units, the plurality of shared memory allreduce work units together perform the allreduce operation on the compute node, wherein the job status object is a data structure in shared memory that indicates, for each of the plurality of shared memory allreduce work units, at least one data element to be processed and a type of processing to be performed on the at least one data element; determining, by an available core of the plurality of processing cores on the compute node, a next shared memory allreduce work unit in the job status object to be processed; and performing, by that available core on the compute node, that next shared memory allreduce work unit. ### Claim 9 (depends on 8) The compute node of claim 8 wherein the computer memory also has disposed within it computer program instructions capable of: copying, by each available core on the compute node into separate shared memory buffers, an array to be reduced by the allreduce operation; and updating, by each available core on the compute node, the job status object with a descriptor of the shared memory buffer into which the available core copied the array. ### Claim 10 (depends on 8) The compute node of claim 8 wherein the computer memory also has disposed within it computer program instructions capable of: providing, by each available core to each other available core, a read-only window into a local memory buffer containing an array to be reduced by the allreduce operation; and updating, by each available core on the compute node, the job status object with a descriptor of the buffer into which the read-only window is provided. ### Claim 11 (depends on 8) The compute node of claim 8 wherein performing, by that available core on the compute node, that next shared memory allreduce work unit further comprises performing a reduction operation on elements of arrays to be reduced. ### Claim 12 (depends on 8) The compute node of claim 8 wherein: the compute node is connected to a plurality of other compute nodes through a data communications network; and performing, by that available core on the compute node, that next shared memory allreduce work unit further comprises transmitting, by that available core, local reduction results to one or more of the other compute nodes through the network. ### Claim 13 (depends on 8) The compute node of claim 8 wherein: the compute node is connected to a plurality of other compute nodes through a data communications network; performing, by that available core on the compute node, that next shared memory allreduce work unit further comprises: receiving global reduction results through the network, and storing the global reduction results into shared memory; and the computer memory also has disposed within it computer program instructions capable of copying, by each available core on the compute node, the global reduction results from the shared memory to local memory of the available core. ### Claim 14 A computer program product for performing an allreduce operation using shared memory, the computer program product disposed upon a computer-readable medium that is not a signal, the computer program product comprising computer program instructions capable of: receiving, by at least one of a plurality of processing cores on a compute node, an instruction to perform an allreduce operation, wherein the plurality of processing cores share the shared memory; establishing, by the core that received the instruction, a job status object that specifies a plurality of shared memory allreduce work units, the plurality of shared memory allreduce work units together perform the allreduce operation on the compute node, wherein the job status object is a data structure in shared memory that indicates, for each of the plurality of shared memory allreduce work units, at least one data element to be processed and a type of processing to be performed on the at least one data element; determining, by an available core of the plurality of processing cores on the compute node, a next shared memory allreduce work unit in the job status object to be processed; and performing, by that available core on the compute node, that next shared memory allreduce work unit. ### Claim 15 (depends on 14) The computer program product of claim 14 wherein establishing, by the core that received the instruction, a job status object for specifying a plurality of shared memory allreduce work units further comprises assigning a plurality of threads for executing the shared memory allreduce work units. ### Claim 16 (depends on 14) The computer program product of claim 14 further comprising computer program instructions capable of: copying, by each available core on the compute node into separate shared memory buffers, an array to be reduced by the allreduce operation; and updating, by each available core on the compute node, the job status object with a descriptor of the shared memory buffer into which the available core copied the array. ### Claim 17 (depends on 14) The computer program product of claim 14 further comprising computer program instructions capable of: providing, by each available core to each other available core, a read-only window into a local memory buffer containing an array to be reduced by the allreduce operation; and updating, by each available core on the compute node, the job status object with a descriptor of the buffer into which the read-only window is provided. ### Claim 18 (depends on 14) The computer program product of claim 14 wherein performing, by that available core on the compute node, that next shared memory allreduce work unit further comprises performing a reduction operation on elements of arrays to be reduced. ### Claim 19 (depends on 14) The computer program product of claim 14 wherein: the compute node is connected to a plurality of other compute nodes through a data communications network; and performing, by that available core on the compute node, that next shared memory allreduce work unit further comprises transmitting, by that available core, local reduction results to one or more of the other compute nodes through the network. ### Claim 20 (depends on 14) The computer program product of claim 14 wherein: the compute node is connected to a plurality of other compute nodes through a data communications network; performing, by that available core on the compute node, that next shared memory allreduce work unit further comprises: receiving global reduction results through the network, and storing the global reduction results into shared memory; and the computer program product further comprises computer program instructions capable of copying, by each available core on the compute node, the global reduction results from the shared memory to local memory of the available core.