US 2026/0010500 A1 · OFFLOADING OF ADAPTIVE ALL REDUCE OPERATIONS

Abstract

Examples described herein relate to a network interface device that includes: a host interface; a direct memory access (DMA) circuitry; a network interface to receive, in at least one packet, time data associated with at least one of multiple layers, wherein the multiple layers provide inputs to a collective operation associated with a large language model (LLM); and circuitry. The circuitry is to based, at least in part, on the time data associated with the multiple layers, identify a first operation of a first layer of the multiple layers as a late completing process relative to times to completion of multiple first operations of other layers and based on the first operation being identified as a late completing process, perform a remedial action to adjust at least one configuration of a first device to execute a second operation of the first layer.

Claims20

Dependent claims

An apparatus comprising:

a network interface device comprising:
a host interface;
a direct memory access (DMA) circuitry;
a network interface to receive, in at least one packet, time data associated with at least one of multiple layers, wherein the multiple layers provide inputs to a collective operation associated with a large language model (LLM);
and
circuitry to:
based, at least in part, on the time data associated with at least one of multiple layers, identify a first operation of a first layer of the multiple layers as a late completing process relative to times to completion of multiple operations of other layers of the multiple layers and
based on the first operation being identified as a late completing process, perform a remedial action to adjust at least one configuration of a first device to execute a second operation of the first layer.

5 more elements

At least one non-transitory computer-readable medium, comprising instructions stored thereon, that if executed by one or more processors, cause the one or more processors to:

configure a network interface device to:
based on identification of a first operation of a first layer as a straggler, perform a remedial action to adjust at least one configuration of circuitry for performance of a second operation of the first layer, wherein the first layer is to provide an input to an collective operation associated with a large language model (LLM).

Dependencies are inferred from the claim text.

Offloading of adaptive all reduce operations

Life of this patent

Offloading of adaptive all reduce operations

Life of this patent

Abstract

Claims20