Sparse optimizations for a matrix accelerator architecture

A method comprising:

• on a general-purpose graphics processor:
• performing a matrix multiply operation on multiple elements of a first matrix and a second matrix via a compute cluster including multiple processing resources coupled with a cache memory, at least one processing resource including a matrix accelerator, wherein the first matrix is a sparse matrix, wherein elements of the first matrix are encoded into a sparse encoding, and wherein the sparse encoding includes a set of non-zero value elements and metadata to identify a location for the non-zero value elements;
• storing the sparse encoding to the cache memory, the cache memory coupled with a compressor to compress the sparse encoding within the cache memory into a compressed representation of the sparse encoding that includes a compressed version of the set of non-zero value elements;
• and
• via the at least one processing resource:
• loading the compressed representation of the sparse encoding from the cache memory;
• decompressing the compressed representation of the sparse encoding into the sparse encoding;
• storing the sparse encoding to a memory within the at least one processing resource;
• loading elements of the second matrix from the cache memory;
• storing the selected elements to the memory within the at least one processing resource, wherein the selected elements of the second matrix correspond with non-zero value elements of the first matrix stored within the sparse encoding and are selected based on the metadata to indicate the locations for the non-zero elements;
• performing the matrix multiply operation on the non-zero value elements from the sparse encoding and the selected elements of the second matrix;
• and
• writing output of the matrix multiply operation to the memory within the at least one processing resource.

A data processing system comprising:

• a memory device;
• and
• a general purpose graphics processor comprising a compute cluster including multiple processing resources coupled with a cache memory, at least one processing resource including a matrix accelerator, the matrix accelerator configured to perform a matrix multiply operation on multiple elements of a first matrix and a second matrix, wherein the first matrix is a sparse matrix, wherein elements of the first matrix are encoded into a sparse encoding, and wherein the sparse encoding includes a set of non-zero value elements and metadata to identify a location for the non-zero value elements;
• wherein the sparse encoding is to be stored to the cache memory;
• wherein the cache memory couples with a compressor to compress the sparse encoding into a compressed representation of the sparse encoding in association with a write the cache memory, wherein the compressed representation of the sparse encoding includes a compressed version of the set of non-zero value elements;
• and
• wherein the at least one processing resource is configured to:
• load the compressed representation of the sparse encoding from the cache memory, decompress the compressed representation of the sparse encoding into the sparse encoding, and store the compressed representation to a memory within the at least one processing resource;
• load selected elements of the second matrix from the cache memory and store the selected elements to the memory within the at least one processing resource, wherein the selected elements of the second matrix correspond with non-zero value elements of the first matrix stored within the compressed representation and are selected based on the metadata to indicate the locations for the non-zero value elements;
• perform the matrix multiply operation on elements from the compressed representation and the selected elements of the second matrix;
• and
• write output of the matrix multiply operation to the memory within the at least one processing resource.