Back to results
US 12,639,253 B1Granted
Methods and systems for allreduce optimization on small tensors
Assigneed-MATRIX CORPORATION
InventorsAayush Ankit, Sudeep Bhoja
Life of this patent
Estimating the term…
Abstract
Methods and systems for allreduce optimization on small tensors. In an embodiment, the present invention provides a system for hierarchical data aggregation includes a first module with interconnected chiplets linked via D2D links, enabling intra-module data sharing and reduce-scatter operations. A second module, also with interconnected chiplets and D2D links, performs a similar intra-module reduce-scatter operation. There are other embodiments as well.
Claims18
A system comprising:
- a first module comprising a first plurality of chiplets;
- a first plurality of die-to-die (D2D) links, the first plurality of chiplets being interconnected via the first plurality of D2D links, the first plurality of chiplets being configured to share a first set of data with each other using the first plurality of D2D links to perform a first set of reduce-scatter operations on the first set of data;
- a second module comprising a second plurality of chiplets;
- a second plurality of D2D links, the second plurality of chiplets being interconnected via the second plurality of D2D links, the second plurality of chiplets being configured to share a second set of data with each other using the second plurality of D2D links to perform a second set of reduce-scatter operations on the second set of data;
- and
- a first peripheral component interconnect express (PCIe) link connecting the first module and the second module, the first plurality of chiplets and the second plurality of chiplets being configured to share a third set of data using the first PCIe link to perform a third set of reduce-scatter operations on the third set of data, the first plurality of chiplets and the second plurality of chiplets being configured to share a fourth set of data using the first PCIe link to perform a first set of allgather operations on the fourth set of data, the first plurality of chiplets further being configured to share a fifth set of data using the first plurality of D2D links to perform a second set of allgather operations on the fifth set of data after completion of the first set of reduce-scatter operations.
3 more elements
A method for performing allreduce operations, the method comprising:
- configure a first module comprising a first plurality of chiplets, the first plurality of chiplets being interconnected via a first plurality of [D2D] die-to-die (D2D) links;
- configure a second module, the second module comprising a second plurality of chiplets, the second plurality of chiplets being interconnected via a second plurality of D2D links;
- sharing a first set of data among the first plurality of chiplets using the first plurality of D2D links;
- performing a first set of reduce-scatter operations by the first plurality of chiplets using the first set of data;
- sharing a second set of data between the first module and a second module via a PCIe link;
- performing a second set of reduce-scatter operations by the first plurality of chiplets and the second plurality of chiplets using the second set of data;
- sharing a third set of data between the first module and a second module via the PCIe link;
- performing a first set of allgather operations by the first plurality of chiplets and the second plurality of chiplets using the third set of data;
- and
- sharing a fourth set of data among the first plurality of chiplets using the first plurality of D2D links;
- and
- performing a second set of allgather operations by the first plurality of chiplets using the fourth set of data after completion of the first set of reduce-scatter operations.
9 more elements
Dependencies are inferred from the claim text.