# US 2021/0279550 A1 · Reduced complexity convolution for convolutional neural networks

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0279550 A1
- Document ID: US20210279550A1
- Status: Published application
- Kind: A1
- Application number: 17/327,988
- Filed: 2021-05-24
- Published: 2021-09-09
- Earliest claimed: 2016-12-27
- Assignees: Texas Instruments Incorporated
- Applicants: Texas Instruments Incorporated
- Inventors: Manu Mathew; Kumar Desappan; Pramod Kumar Swami
- Web page: https://patentgrep.com/patent/US20210279550A1

## CPC

- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06F 7/5443: {Sum of products (for applications thereof, see the relevant places, e.g. G06F17/10, H03H17/00)}
- G06F 17/15: Correlation function computation {including computation of convolution operations (arithmetic circuits for sum of products per se, e.g. multiply-accumulators G06F7/5443; digital filters, e.g. FIR, IIR, adaptive filters H03H17/00)}
- G06F 17/153: {Multidimensional correlation or convolution}
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/09: Supervised learning
- G06N 3/10: Interfaces, programming languages or software development kits, e.g. for simulating neural networks
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A method for convolution in a convolutional neural network (CNN) is provided that includes accessing a coefficient value of a filter corresponding to an input feature map of a convolution layer of the CNN, and performing a block multiply accumulation operation on a block of data elements of the input feature map, the block of data elements corresponding to the coefficient value, wherein, for each data element of the block of data elements, a value of the data element is multiplied by the coefficient value and a result of the multiply is added to a corresponding data element in a corresponding output block of data elements comprised in an output feature map.

## Claims

### Claim 1

A method comprising:
training a convolutional neural network (CNN) to generate coefficients associated with a layer of the CNN;
tuning the layer of the CNN, wherein the tuning includes setting selected coefficients to zero; and
for each respective nonzero coefficient of the coefficients, perform a block multiply accumulation (BMA) operation, wherein the BMA operation multiplies the respective nonzero coefficient with an input data block corresponding to the respective nonzero coefficient.

### Claim 2 (depends on 1)

The method of claim 1, wherein:
the training is based on a stochastic gradient descent back propagation.

### Claim 3 (depends on 1)

The method of claim 1, wherein:
the training includes a weight decay configured to induce sparsity in the selected coefficients.

### Claim 4 (depends on 1)

The method of claim 1, wherein:
the BMA operation accumulates a result in an output data block corresponding to the respective nonzero coefficient.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
determining a sparse coefficient map for the layer of the CNN, wherein the sparse coefficient map identifies the selected coefficients that are set to zero.

### Claim 6 (depends on 1)

The method of claim 1, wherein:
the selected coefficients are selected based on a threshold determined for the layer of the CNN

### Claim 7 (depends on 6)

The method of claim 6 wherein:
the threshold for the layer of the CNN is based on a sparsity target and a maximum threshold value.

### Claim 8 (depends on 7)

The method of claim 7, wherein:
the sparsity target is a ratio of a number of zero coefficients to a total number of coefficients of the layer of the CNN.

### Claim 9 (depends on 7)

The method of claim 7, wherein:
the maximum threshold value is based on a maximum coefficient value; and
the maximum coefficient value is a highest absolute value of the selected filter coefficients for the layer of the CNN.

### Claim 10 (depends on 7)

The method of claim 7, wherein:
the sparsity target is based on a number of layers of the CNN.

### Claim 11

An apparatus comprising:
a memory storing software instructions; and
one or more processors coupled to the memory, the one or more processors being operable when executing the software instructions to:
train a convolutional neural network (CNN) to generate coefficients associated with a layer of the CNN;
tune the layer of the CNN, wherein the tuning includes setting selected coefficients to zero; and
for each respective nonzero coefficient of the coefficients, perform a block multiply accumulation (BMA) operation, wherein the BMA operation multiplies the respective nonzero coefficient with an input data block corresponding to the respective nonzero coefficient.

### Claim 12 (depends on 11)

The apparatus of claim 11, wherein:
the software instructions to train include training based on a stochastic gradient descent back propagation.

### Claim 13 (depends on 11)

The apparatus of claim 11, wherein:
the software instructions to train include a weight decay configured to induce sparsity in the selected coefficients.

### Claim 14 (depends on 11)

The apparatus of claim 11, wherein:
the BMA operation accumulates a result in an output data block corresponding to the respective nonzero coefficient.

### Claim 15 (depends on 11)

The apparatus of claim 11, further comprising:
software instructions to determine a sparse coefficient map for the layer of the CNN, wherein the sparse coefficient map identifies the selected coefficients that are set to zero.

### Claim 16 (depends on 11)

The apparatus of claim 11, wherein:
the selected coefficients are selected based on a threshold determined for the layer of the CNN

### Claim 17 (depends on 16)

The apparatus of claim 16 wherein:
the threshold for the layer of the CNN is based on a sparsity target and a maximum threshold value.

### Claim 18 (depends on 17)

The apparatus of claim 17, wherein:
the sparsity target is a ratio of a number of zero coefficients to a total number of coefficients of the layer of the CNN.

### Claim 19 (depends on 17)

The apparatus of claim 17, wherein:
the maximum threshold value is based on a maximum coefficient value; and
the maximum coefficient value is a highest absolute value of the selected filter coefficients for the layer of the CNN.

### Claim 20 (depends on 17)

The apparatus of claim 17, wherein:
the sparsity target is based on a number of layers of the CNN.