Collectives-aware load balancing

A system comprising:

• one or more processing devices;
• and
• memory communicatively coupled with and readable by the one or more processing devices and having stored therein processor-readable instructions which, when executed by the one or more processing devices, cause the system to perform operations comprising:
• determining metadata associated with a collective operation, wherein the metadata specifies at least one of:
• a job identifier corresponding to a unit of work to be completed in conjunction with the collective operation;
• a collective type of the collective operation;
• and/or
• an ordering mode for packets corresponding to the collective operation;
• sending the metadata associated with the collective operation to a network interface card (NIC) communicatively coupled to the system;
• and
• causing the NIC to transmit a data packet with the metadata embedded in a cookie of the data packet to a switch of a network fabric to cause the switch to use a selected network path and/or selected load-balancing for the collective operation based at least in part on one or more of the job identifier, the collective type, and/or the ordering mode.

One or more non-transitory, machine-readable media having machine-readable instructions thereon which, when executed by one or more processing devices, cause a system to perform operations comprising:

• determining metadata associated with a collective operation, wherein the metadata specifies at least one of:
• a job identifier corresponding to a unit of work to be completed in conjunction with the collective operation;
• a collective type of the collective operation;
• and/or
• an ordering mode for packets corresponding to the collective operation;
• sending the metadata associated with the collective operation to a network interface card (NIC) communicatively coupled to the system;
• and
• causing the NIC to transmit a data packet with the metadata embedded in a cookie of the data packet to a switch of a network fabric to cause the switch to use a selected network path and/or selected load-balancing for the collective operation based at least in part on one or more of the job identifier, the collective type, and/or the ordering mode.