Workload mapping to matrix multiplication engines

A method comprising:

• operating compute tiles to perform tensor computations using per‑tile compute engines that internally accumulate partial results to completion;
• managing a memory addressing scheme using a system address decoder that provides tile‑private address ranges for data used by a given tile and tile‑shared address ranges for data used by multiple tiles;
• executing a first parallelization mode in which a parameterized portion of a model dataset is distributed among tiles and input data is made available across tiles, the mode advancing work across tiles;
• executing a second parallelization mode in which tiles produce partial results for a common output address space and propagate the partial results among neighbor tiles to an owner tile that completes accumulation and commits the output;
• and
• performing a fused execution across projection and gating operations to maintain a steady state and avoid growth of intermediate activation tensors while overlapping data movement and computation within a per‑tile memory budget.

An apparatus comprising:

• a processing system including one or more processors;
• a plurality of compute tiles;
• a memory subsystem;
• and
• an interconnect arranged to communicate among the compute tiles;
• the processors configured to:
• operate the compute tiles to perform tensor computations using per‑tile compute engines that internally accumulate partial results to completion;
• manage a memory addressing scheme using a system address decoder that provides tile‑private address ranges for data used by a given tile and tile‑shared address ranges for data used by multiple tiles;
• execute a first parallelization mode in which a parameterized portion of a model dataset is distributed among tiles and input data is made available across tiles, the mode advancing work across tiles;
• execute a second parallelization mode in which tiles produce partial results for a common output address space and propagate the partial results among neighbor tiles to an owner tile that completes accumulation and commits the output;
• and
• perform a fused execution across projection and gating operations to maintain a steady state and avoid growth of intermediate activation tensors while overlapping data movement and computation within a per‑tile memory budget.