Back to results
US 2025/0053613 A1Published application
Random sparsity handling in a systolic array
AssigneeIntel Corporation
InventorsChunhui Mei, Hong Jiang, Jiasheng Chen, Yongsheng Liu, Yan Li
Life of this patent
Estimating the term…
Abstract
Matrix multiply units can take advantage of input sparsity by zero gating ALUs, which saves power consumption, but compute throughput does not increase. To improve compute throughput from sparsity, processing resources in a matrix accelerator can skip computation with zero involved in input or output. If zeros in input can be skipped, the processing units can focus calculations on generating meaningful non-zero output.
Claims21
1-20. (canceled)
A graphics processor comprising:
- a plurality of compute resources;
- a matrix accelerator;
- and
- a parallel processing engine configured to perform graphics and compute operations via the plurality of compute resources and the matrix accelerator, the matrix accelerator configured to:
- load elements of a first submatrix and a second submatrix into memory accessible to the matrix accelerator, the first submatrix and the second submatrix having unstructured sparsity;
- merge an element of the first submatrix into the second submatrix to increase sparsity of the first submatrix and reduce the sparsity of the second submatrix, the element merged according to a merge pattern determined for groupings of elements of the first submatrix and the second submatrix;
- generate metadata to indicate the merge pattern;
- and
- provide the second submatrix and the metadata to an array of processing resources within the matrix accelerator as input for a matrix operation.
6 more elements
A method comprising:
- reading two or more tiles of elements of a first matrix;
- merging a first group of elements in a first tile of the two or more tiles of elements of the first matrix into a second group of elements of a corresponding group of elements in a second tile of the two or more tiles of elements;
- reading two or more tiles of elements of a second matrix;
- and
- performing a matrix multiply operation having input including the second group of elements in the second tile and selected elements from the two or more tiles of elements of the second matrix.
2 more elements
A data processing system comprising:
- a memory device;
- and
- an accelerator device coupled with the memory device, the accelerator device including a matrix accelerator configured to:
- load elements of a first submatrix and a second submatrix into memory accessible to the matrix accelerator, the first submatrix and the second submatrix having unstructured sparsity;
- merge an element of the first submatrix into the second submatrix to increase sparsity of the first submatrix and reduce the sparsity of the second submatrix, the element merged according to a merge pattern determined for groupings of elements of the first submatrix and the second submatrix;
- generate metadata to indicate the merge pattern;
- and
- provide the second submatrix and the metadata to an array of processing resources within the matrix accelerator as input for a matrix operation.
5 more elements
Dependencies are inferred from the claim text.