# US 2019/0244086 A1 · Configurable neural network engine for convolutional filter sizes

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0244086 A1
- Document ID: US20190244086A1
- Status: Published application
- Kind: A1
- Application number: 16/234,184
- Filed: 2018-12-27
- Published: 2019-08-08
- Earliest claimed: 2018-02-08
- Assignees: Western Digital Technologies, Inc.
- Applicants: Western Digital Technologies, Inc.
- Inventors: Luiz M. Franca-Neto
- Web page: https://patentgrep.com/patent/US20190244086A1

## CPC

- G06N 3/063: using electronic means
- G06F 18/21: Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F 18/22: Matching criteria, e.g. proximity measures
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/09: Supervised learning
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/82: using neural networks
- G06V 10/94: Hardware or software architectures specially adapted for image or video understanding
- G06V 10/955: {using specific electronic processors}
- G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques}
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Some embodiments include a special-purpose hardware accelerator that can perform specialized machine learning tasks during both training and inference stages. For example, this hardware accelerator uses a systolic array having a number of data processing units (“DPUs”) that are each connected to a small number of other DPUs in a local region. Data from the many nodes of a neural network is pulsed through these DPUs with associated tags that identify where such data was originated or processed, such that each DPU has knowledge of where incoming data originated and thus is able to compute the data as specified by the architecture of the neural network. These tags enable the systolic neural network engine to perform computations during backpropagation, such that the systolic neural network engine is able to support training.

## Claims

### Claim 1

A device for performing computations of a convolutional neural network, the device comprising:
a processing chip including:
a first arrangement of a plurality of tensor arrays including circuitry configured for performing computations according to a default convolutional filter size;
a second arrangement of a plurality of memory cells configured to store outputs of corresponding ones of the tensor arrays;
a plurality of interconnects connecting particular ones of the tensor arrays to particular ones of the memory cells;
a computer-readable memory storing instructions for configuring the processing chip to perform computations of the convolutional neural network; and
a controller configured by the instructions to:
determine, for a particular convolution of a convolutional layer of the convolutional neural network, a particular convolutional filter size used for the particular convolution;
when the particular convolutional filter size equals the default convolutional filter size, configure a processing unit to include one of the tensor arrays, and configure the processing unit to perform the particular convolution using the default convolutional filter size;
when the particular convolutional filter size is less than the default convolutional filter size, configure the processing unit to include one of the tensor arrays, and configure the processing unit to perform the particular convolution using the default convolutional filter size padded with zeros such that a padded portion with an unpadded portion of the default convolutional filter corresponds to the particular convolutional filter size; and
when the particular convolutional filter size is greater than the default convolutional filter size, configure the processing unit to include multiple tensor arrays to perform the particular convolution.

### Claim 2 (depends on 1)

The device of claim 1, wherein the default convolutional filter size is 5×5×1.

### Claim 3 (depends on 1)

The device of claim 1, wherein the device is configured to:
provide input into the processing chip for the processing chip to perform the particular convolution using the processing unit; and
provide an output of the processing chip as an output of the convolutional neural network.

### Claim 4 (depends on 1)

The device of claim 1, wherein the controller is further configured by the instructions to configure a particular tensor array to perform a computation of a fully connected layer of the convolutional neural network by instructing the tensor array to use a center value of the default convolutional filter for processing input data and pad remaining values of the default convolutional filter with zeros.

### Claim 5 (depends on 1)

The device of claim 1, wherein at least one tensor array of the plurality of tensor arrays includes circuitry to perform a single multiplication operation.

### Claim 6 (depends on 1)

The device of claim 1, wherein at least one tensor array of the plurality of tensor arrays includes circuitry to perform a plurality of multiplication operations.

### Claim 7 (depends on 1)

The device of claim 1, wherein the controller is further configured by the instructions to configure the processing chip into a plurality of processing units that collectively perform the computations of multiple layers of the convolutional neural network.

### Claim 8 (depends on 7)

The device of claim 7, wherein the plurality of processing units form an array, wherein the array comprises a plurality of systolic transfer structures to systolically transfer outputs generated by a first subset of the processing units for one layer of the convolutional neural network to a second subset of processing units assigned to a next layer of the convolutional neural network.

### Claim 9

A method for performing computations of a neural network, the method comprising:
identifying a default filter size of a plurality of tensor arrays included in a processing chip, wherein the processing chip further includes a plurality of memory cells for storing outputs of corresponding ones of the tensor arrays and a plurality of interconnects connecting particular ones of the tensor arrays to particular ones of the memory cells;
determining, for a particular set of operations of a layer of the neural network, a particular filter size used for the particular set of operations;
determining that a particular filter size is greater than the default filter size; and
in response to determining that the particular filter size is greater than the default filter size, configuring a processing unit to include multiple tensor arrays to perform the particular set of operations.

### Claim 10 (depends on 9)

The method of claim 9, wherein the default filter size is 5×5×1.

### Claim 11 (depends on 9)

The method of claim 9, wherein the method further comprises:
providing input into the processing chip for the processing chip to perform the particular set of operations using the processing unit; and
providing an output of the processing chip as an output of the neural network.

### Claim 12 (depends on 9)

The method of claim 9, wherein the method further comprises configuring a particular tensor array to perform a computation of a fully connected layer of the neural network by instructing the tensor array to use a center value of the default filter for processing input data and padding remaining values of the default filter with zeros.

### Claim 13 (depends on 9)

The method of claim 9, wherein at least one tensor array of the plurality of tensor arrays includes circuitry to perform a single multiplication operation.

### Claim 14 (depends on 9)

The method of claim 9, wherein at least one tensor array of the plurality of tensor arrays includes circuitry to perform a plurality of multiplication operations.

### Claim 15 (depends on 9)

The method of claim 9, wherein the method further comprises configuring the processing chip into a plurality of processing units that collectively perform the computations of multiple layers of the neural network.

### Claim 16 (depends on 15)

The method of claim 15, wherein the plurality of processing units form an array, wherein the array comprises a plurality of systolic transfer structures to systolically transfer outputs generated by a first subset of the processing units for one layer of the neural network to a second subset of processing units assigned to a next layer of the neural network.

### Claim 17

A controller comprising one or more processors configured to:
identify a default filter size of a plurality of tensor arrays included in a processing chip, wherein the processing chip further includes a plurality of memory cells for storing outputs of corresponding ones of the tensor arrays and a plurality of interconnects connecting particular ones of the tensor arrays to particular ones of the memory cells;
determine, for a particular set of operations of a layer of a neural network, a particular filter size used for the particular set of operations;
when the particular filter size equals the default filter size, configure a processing unit to include one of the tensor arrays, and configure the processing unit to perform the particular set of operations using the default filter size;
when the particular filter size is less than the default filter size, configure the processing unit to include one of the tensor arrays, and configure the processing unit to perform the particular set of operations using the default filter size padded with zeros such that an unpadded portion of the default filter corresponds to the particular filter size; and
when the particular filter size is greater than the default filter size, configure the processing unit to include multiple tensor arrays to perform the particular set of operations.

### Claim 18 (depends on 17)

The controller of claim 17, wherein the default filter size is 5×5×1.

### Claim 19 (depends on 17)

The controller of claim 17, wherein the one or more processors are further configured to:
provide input into the processing chip for the processing chip to perform the particular set of operations using the processing unit; and
provide an output of the processing chip as an output of the neural network.

### Claim 20 (depends on 17)

The controller of claim 17, wherein the one or more processors are further configured for a particular tensor array to perform a computation of a fully connected layer of the neural network by instructing the tensor array to use a center value of the default filter for processing input data and padding remaining values of the default filter with zeros, wherein the controller performs the operations by computer executable instructions stored in a non-transitory computer-readable medium.