Sparse matrix multiplication acceleration mechanism
AssigneeIntel Corporation
InventorsSubramaniam Maiyuran, Mathew Nevin, Jorge Parra, Ashutosh Garg, Shubra Marwaha, Shubh Shah
Life of this patent
Estimating the term…
Abstract
An apparatus to facilitate acceleration of matrix multiplication operations. The apparatus comprises a systolic array including matrix multiplication hardware to perform multiply-add operations on received matrix data comprising data from a plurality of input matrices and sparse matrix acceleration hardware to detect zero values in the matrix data and perform one or more optimizations on the matrix data to reduce multiply-add operations to be performed by the matrix multiplication hardware.
Claims20
An apparatus comprising:
- a systolic multiplier, including:
- a first set of first in first out (FIFO) buffers to store data in a first input matrix;
- a second set of FIFO buffers to store data in a second input matrix;
- a plurality of processing elements (PEs) to receive the matrix data from the first set of FIFO buffers and the second set of FIFO buffers and perform multiply-add operations on first input matrix and the second input matrix;
- a plurality of storage elements to locally store intermediate matrix multiplication values;
- and
- sparse matrix acceleration circuitry to detect zero values in the matrix data and perform one or more optimizations on the matrix data to reduce the multiply-add operations to be performed by the matrix systolic multiplier, including swapping rows with other rows in a sub-matrix for each of a plurality of sub-matrices of the first input matrix.
A method comprising:
- receiving data in a first input matrix at a systolic multiplier;
- receiving data in a second input matrix at the systolic multiplier;
- detecting, at the systolic multiplier, zero values in the first matrix data;
- performing, at the systolic multiplier, one or more optimizations on the matrix data to reduce multiply-add operations to be performed, including swapping rows with other rows in a sub-matrix for each of a plurality of sub-matrices of the first input matrix;
- and
- performing, at the systolic multiplier, the multiply-add operations on the optimized matrix data at matrix multiplication circuitry.
A graphics processing unit (GPU) comprising:
- a first set of first in first out (FIFO) buffers to store data in a first input matrix;
- a second set of FIFO buffers to store data in a second input matrix;
- a plurality of processing elements (PEs) to receive the matrix data from the first set of FIFO buffers and the second set of FIFO buffers and perform multiply-add operations on first input matrix and the second input matrix;
- and
- a plurality of storage elements to locally store intermediate matrix multiplication values;
- and
- sparse matrix acceleration circuitry to detect zero values in the matrix data and perform one or more optimizations on the matrix data to reduce multiply-add operations to be performed by the matrix systolic multiplier, including swapping rows with other rows in a sub-matrix for each of a plurality of sub-matrices of the first input matrix.
Dependencies are inferred from the claim text.