# US 2020/0401895 A1 · Neural network hardware accelerator system with zero-skipping and hierarchical structured pruning methods

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0401895 A1
- Document ID: US20200401895A1
- Status: Published application
- Kind: A1
- Application number: 16/449,009
- Filed: 2019-06-21
- Published: 2020-12-24
- Earliest claimed: 2019-06-21
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: SHAI LITVAK; EYAL HOCHBERG
- Web page: https://patentgrep.com/patent/US20200401895A1

## CPC

- G06F 17/153: {Multidimensional correlation or convolution}
- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- G06F 7/5443: {Sum of products (for applications thereof, see the relevant places, e.g. G06F17/10, H03H17/00)}
- G06F 17/15: Correlation function computation {including computation of convolution operations (arithmetic circuits for sum of products per se, e.g. multiply-accumulators G06F7/5443; digital filters, e.g. FIR, IIR, adaptive filters H03H17/00)}
- G06N 3/02: Neural networks
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/063: using electronic means
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/09: Supervised learning
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Systems and methods for structured-pruning, zero-skipping and accelerated processing of an artificial neural network (ANN) are described. The ANN may include one or more convolution layers. 2D channels in filters of the convolution layers comprise fully pruned channels (FPCs), each containing only zero weights, and mixed channels (MCs), each containing at least one non-zero weight. At least a portion of the MCs satisfy a limited zero sequence (LZS) condition limiting the number and location of zeroes in the MC. The LZS condition may be based on a number of weights that a zero-skipping circuit of a computing system for processing the ANN is configured to evaluate and skip in a single cycle. Thus, when processing the structurally-pruned ANN using the zero-skipping method, the computing system may avoid processing zero weights. This may allow speeding up the ANN processing and reducing the power required for processing.

## Claims

### Claim 1

A computing system for processing an artificial neural network (ANN), comprising:
a processor comprising a zero-skipping circuit configured to locate non-zero weights;
a memory; and
an ANN stored within the memory, wherein the ANN comprises a plurality of layers including one or more convolution layers, wherein each of the one or more convolution layers comprises a plurality of filters, each filter comprises a plurality of channels, each channel comprises a plurality of rows, and each row comprises a plurality of weights,
wherein the plurality of channels in the one or more convolution layers comprises one or more fully pruned channel (FPCs) and one or more mixed 2D channel (MCs), wherein each of the one or more FPCs comprises only zero weights and each of the one or more MCs comprises at least one non-zero weight, and
wherein at least a portion of the one or more MCs satisfy a limited zero sequence (LZS) condition based on a number of weights the zero-skipping circuit is configured to process in a single cycle.

### Claim 2 (depends on 1)

The computing system of claim 1, wherein:
the LZS condition comprises a bounded length zeroes (BZS) condition that imposes a maximum length on zero sequences.

### Claim 3 (depends on 1)

The computing system of claim 1, wherein:
the LZS condition comprises a maximum number of rows over which zero sequences extend.

### Claim 4 (depends on 3)

The computing system of claim 3, wherein:
the maximum number of rows is two.

### Claim 5 (depends on 1)

The computing system of claim 1, wherein:
the LZS condition is based on a maximal number and a location of zero weights in a sequence of filter weights the zero-skipping circuit is configured to process and skip in a single cycle.

### Claim 6 (depends on 1)

The computing system of claim 1, wherein:
the one or more convolution layers comprises at least 33% zero weights.

### Claim 7 (depends on 1)

The computing system of claim 1, wherein:
the one or more FPCs comprise at least 20% of channels in the one or more convolution layers.

### Claim 8 (depends on 1)

The computing system of claim 1, wherein:
the portion of the one or more MCs comprise at least 95% of the one or more MCs.

### Claim 9 (depends on 1)

The computing system of claim 1, wherein:
the LZS condition is based at least in part on a scanning order of the computing system.

### Claim 10 (depends on 9)

The computing system of claim 9, wherein:
the LZS condition applies to sequences of consecutive zero weights over more than one channel according to the scanning order of the computing system.

### Claim 11 (depends on 1)

The computing system of claim 1, wherein:
the ANN further comprises one or more fully connected layers that satisfy the LZS condition.

### Claim 12

A method for pruning an artificial neural network (ANN), comprising:
identifying a target ratio of zero weights for a layer of the ANN;
identifying a limited zero sequence (LZS) condition for the layer, wherein the LZS condition comprises a limitation on a location or sequence length of zeroes in the layer;
pruning the layer based at least in part on the target ratio of zero weights by setting one or more weights of the layer to zero;
determining whether each sequence of zero weights in the layer violates the LZS condition after the pruning; and
resetting at least one weight within each sequence of zero weights that violates the LZS condition to a non-zero value.

### Claim 13 (depends on 12)

The method of claim 12, further comprising:
generating a zero mask based at least in part on the pruning and the resetting; and
continuing to train the ANN based on the zero mask.

### Claim 14 (depends on 12)

The method of claim 12, wherein:
the LZS condition comprises a bounded length zeroes (BZS) condition comprising a maximum length of zero sequences.

### Claim 15 (depends on 12)

The method of claim 12, wherein:
the LZS condition comprises a limited hardware prediction (LHP) condition based on sequences of zero weights within a maximal number of rows of weights in a filter that a zero-skipping circuit of a computing system for processing the ANN is configured to process and skip in a single cycle.

### Claim 16 (depends on 12)

The method of claim 12, further comprising:
identifying a target ratio of fully pruned channel (FPCs) for the layer of the ANN, wherein FPCs contain only zero weights, and wherein the pruning is further based on the target ratio of FPCs.

### Claim 17 (depends on 12)

The method of claim 12, wherein:
at least 95% of mixed 2D channel (MCs) within the layer satisfy the LZS after resetting the at least one weight.

### Claim 18

A computing system for processing an artificial neural network (ANN), comprising:
one or more memory units configured to store weights of the ANN and values of an input tensor; and
a plurality of accelerator sub-systems, wherein each accelerator subsystem further comprises:
a multiplexing buffer configured to store weights of the ANN;
a look-ahead buffer configured to store additional weights of the ANN;
a zero-skipping circuit configured to identify a next non-zero weight of the ANN; and
a plurality of multiply-accumulate (MAC) units configured to simultaneously process the next non-zero weight identified by the zero-skipping circuit together with a plurality of corresponding values from the input tensor.

### Claim 19 (depends on 18)

The computing system of claim 18, wherein:
the ANN comprises at least one convolution layer, the at least one convolution layer comprises a plurality of filters, each filter comprises a plurality of channels, each channel comprises a plurality of rows, and each row comprises a plurality of weights, and wherein the look-ahead buffer is configured to store a fixed number of rows of weights.

### Claim 20 (depends on 18)

The computing system of claim 18, further comprising:
a control unit configured to identify and skip fully pruned channel (FPCs) of the ANN.

### Claim at position 21

21-25. (canceled)