US 12,762,394 B1 · Managing data sparsity for neural networks

Abstract

Approaches in accordance with various embodiments provide for the processing of sparse matrices for mathematical and programmatic operations. In particular, various embodiments enforce sparsity constraints for performing sparse matrix multiply-add instruction (MMA) operations. Deep neural networks can exhibit significant sparsity in the data used in operations, both in the activations and weights. The computational load can be reduced by excluding zero-valued data elements. A sparsity constraint is applied across all submatrices of a sparse matrix, providing fine-grained structured sparsity that is evenly distributed across the matrix. The matrix may then be compressed since a minimum number of elements of the matrix are known to have zero value. Matrix operations are then performed using these matrices.

Claims20

Dependent claims

A processor, comprising:

one or more circuits to;
identify a matrix multiplication operation subject to a sparsity constraint indicating a required minimum number of zero-values for a sparsity pattern repeated across one or more matrix operands of the matrix multiplication operation;
and
responsive to the one or more matrix operands satisfying the sparsity constraint,
compress the one or more matrix operands of the matrix multiplication operation to exclude a plurality of zero-values from the one or more matrix operands according to the sparsity pattern.

2 more elements

Dependencies are inferred from the claim text.

Facts

Application

17/866,031Filed 15 Jul 2022 · continuation

Priority

2 May 2018

Granted

13 Oct 2026

Term ends

Not estimatedThe source records don’t support an estimate.

Classification5 codes

G06N 3/08
Neural networks: learning methods
G06F 17/16
Complex mathematical operations: matrix or vector computation , e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
G06N 3/04
Neural networks: architecture, e.g. interconnection topology
G06N
AI and machine learning

Managing data sparsity for neural networks

Life of this patent

Managing data sparsity for neural networks

Life of this patent

Abstract

Claims20