Efficient iterative collective operations using a network-attached memory

A computer system, comprising:

• a processor;
• and
• a storage device storing instructions to:
• receive, by a first computing node of a plurality of computing nodes from a processing entity in the first computing node, a first request to perform a collective operation, the first request indicating a first virtual address and a first data unit to write to a first segment of a memory region, and the processing entity comprising a rank associated with a program under execution in the computing node;
• perform a memory allocation process by allocating, to the rank, the memory region based on the first virtual address and determining a descriptor for a physical location of the memory;
• store, in a data structure, a mapping of the first virtual address to the descriptor;
• perform the collective operation, the collective operation comprising at least writing the first data unit to the first segment and accessing data units from other segments of the memory region;
• receive a second request to perform an update operation, the second request indicating:
• a second virtual address;
• a number of one or more portions of a memory region segment to be updated;
• a list of indices corresponding to the one or more portions of the memory region segment to be updated, wherein a respective index corresponds to an ordered portion of the memory region segment;
• and
• the corresponding data units to write to the one or more portions, wherein a respective corresponding data unit includes a new value to be updated at the corresponding respective index;
• search a data structure based on the second virtual address to obtain the mapping of the first virtual address to the descriptor, the second virtual address matching the first virtual address;
• and
• update, based on the obtained mapping, the one or more portions of the memory region segment by writing the corresponding data units to the one or more portions.

A non-transitory computer-readable storage medium storing instructions to:

• receive, by a first computing node from a processing entity in the first computing node, a first request to perform a collective operation, the first request indicating a first virtual address, and the processing entity comprising a rank associated with a program under execution in the computing node;
• store, in a data structure, a mapping of the first virtual address to a descriptor for a physical location of a memory region allocated for performance of the collective operation;
• perform the collective operation, which includes writing a first data unit to a first segment of the memory region and accessing data units from other segments of the memory region;
• receive a second request to perform an update operation, the second request indicating:
• a second virtual address:
• a number of one or more portions of a memory region segment to be updated;
• a list of indices corresponding to the one or more portions of the memory region segment to be updated, wherein a respective index corresponds to an ordered portion of the memory region segment;
• and
• one or more new values to write to the one or more portions;
• search a data structure based on the second virtual address to obtain the mapping of the first virtual address to the descriptor, the second virtual address matching the first virtual address;
• and
• update, based on the obtained mapping, the one or more portions of the memory region segment by writing the one or more new values to the one or more portions at the indicated indices.