# US 2019/0244085 A1 · Configurable neural network engine of tensor arrays and memory cells

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0244085 A1
- Document ID: US20190244085A1
- Status: Published application
- Kind: A1
- Application number: 16/233,876
- Filed: 2018-12-27
- Published: 2019-08-08
- Earliest claimed: 2018-02-08
- Assignees: Western Digital Technologies, Inc.
- Applicants: Western Digital Technologies, Inc.
- Inventors: Luiz M. Franca-Neto
- Web page: https://patentgrep.com/patent/US20190244085A1

## CPC

- G06N 3/063: using electronic means
- G06F 18/21: Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F 18/22: Matching criteria, e.g. proximity measures
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/09: Supervised learning
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/82: using neural networks
- G06V 10/94: Hardware or software architectures specially adapted for image or video understanding
- G06V 10/955: {using specific electronic processors}
- G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques}
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Some embodiments include a special-purpose hardware accelerator that can perform specialized machine learning tasks during both training and inference stages. For example, this hardware accelerator uses a systolic array having a number of data processing units (“DPUs”) that are each connected to a small number of other DPUs in a local region. Data from the many nodes of a neural network is pulsed through these DPUs with associated tags that identify where such data was originated or processed, such that each DPU has knowledge of where incoming data originated and thus is able to compute the data as specified by the architecture of the neural network. These tags enable the systolic neural network engine to perform computations during backpropagation, such that the systolic neural network engine is able to support training.

## Claims

### Claim 1

A device for performing computations of a convolutional neural network, the device comprising:
a processing chip including:
a first arrangement of a plurality of tensor arrays;
a second arrangement of a plurality of memory cells;
a plurality of interconnects connecting particular ones of the tensor arrays to particular ones of the memory cells;
a computer-readable memory storing instructions for configuring the processing chip to perform computations of the convolutional neural network; and
a controller configured by the instructions to:
determine, for a particular layer of the convolutional neural network, a number of input channels and a number of output maps generated for particular ones of a plurality of pixels;
configure a portion of the processing chip into a plurality of processing units, wherein a particular processing unit performs computations associated with a particular one of the plurality of pixels; and wherein individual ones of the plurality of processing units include:
a number of the tensor arrays determined based on the number of input channels, and
a number of memory cells corresponding to the number of output maps; and
assign the plurality of processing units to perform computations of the particular layer.

### Claim 2 (depends on 1)

The device of claim 1, wherein the device is configured to provide input into the processing chip, wherein a corresponding one of the processing units are configured to perform the computations of the particular layer for individual ones of the plurality of pixels using the number of tensor arrays and to store output using the number of memory cells, the device further configured to provide an output of the processing chip as an output of the convolutional neural network.

### Claim 3 (depends on 1)

The device of claim 1, wherein for a particular processing unit, the tensor arrays of the number of tensor arrays are configured to send output data to corresponding ones of the number of memory cells over a subset of the interconnects.

### Claim 4 (depends on 1)

The device of claim 1, wherein a default processing unit includes one tensor array and one memory cell connected to one another by one interconnect, and wherein the default processing unit has capacity to process one channel and generate one output.

### Claim 5 (depends on 4)

The device of claim 4, wherein the controller is further configured by the instructions to:
determine that the number of input channels for a particular one of the plurality of pixels in the particular layer exceeds the capacity of the default processing unit, and
configure a corresponding one of the plurality of processing units to combine tensor arrays of multiple default processing units while using less than all of the memory cells of the plurality of processing units.

### Claim 6 (depends on 4)

The device of claim 4, wherein the controller is further configured by the instructions to:
determine that the number of output maps generated for a particular one of the plurality of pixels in the particular layer exceeds the capacity of the default processing unit, and
configure a corresponding one of the plurality of processing units to combine the memory cells of multiple default processing units while using less than all of the tensor arrays of the plurality of processing units.

### Claim 7 (depends on 1)

The device of claim 1, wherein at least one tensor array of the plurality of tensor arrays includes circuitry to perform a single multiplication operation.

### Claim 8 (depends on 1)

The device of claim 1, wherein at least one tensor array of the plurality of tensor arrays includes circuitry to perform a plurality of multiplication operations.

### Claim 9 (depends on 1)

The device of claim 1, wherein the controller is further configured by the instructions to configure the processing chip into a plurality of processing units that collectively perform the computations of multiple layers of the convolutional neural network.

### Claim 10 (depends on 9)

The device of claim 9, wherein the plurality of processing units form an array, wherein the array comprises a plurality of systolic transfer structures to systolically transfer output maps generated by a first subset of the processing units for one layer of the convolutional neural network to a second subset of processing units assigned to a next layer of the convolutional neural network.

### Claim 11

A method for performing computations of a neural network, the method comprising:
determining, for a particular layer of the neural network, a number of input channels and a number of output maps generated for particular ones of a plurality of pixels;
configuring a portion of a processing chip into a plurality of processing units, wherein the processing chip includes one or more tensor arrays and one or more memory cells, wherein a particular processing unit performs computations associated with a particular one of the plurality of pixels; and wherein individual ones of the plurality of processing units include:
a number of the tensor arrays determined based on the number of input channels, and
a number of memory cells corresponding to the number of output maps; and
assign the plurality of processing units to perform computations of the particular layer.

### Claim 12 (depends on 11)

The method of claim 11, further comprising:
providing input into the processing chip to perform the computations of the particular layer for individual ones of the plurality of pixels using the number of tensor arrays and to store output using the number of memory cells, and
providing an output of the processing chip as an output of the neural network.

### Claim 13 (depends on 11)

The method of claim 11, wherein for a particular processing unit, the method further comprises sending, by the tensor arrays of the number of tensor arrays, output data to corresponding ones of the number of memory cells over a subset of the interconnects connecting particular ones of the tensor arrays to particular ones of the memory cells.

### Claim 14 (depends on 11)

The method of claim 11, wherein a default processing unit includes one tensor array and one memory cell connected to one another by one interconnect connecting the one tensor array to the one memory cell, and wherein the method further comprises processing one channel and generate one output using the default processing unit.

### Claim 15 (depends on 14)

The method of claim 14, wherein the method further comprises:
determining that the number of input channels for a particular one of the plurality of pixels in the particular layer exceeds a capacity of the default processing unit, and
configuring a corresponding one of the plurality of processing units to combine tensor arrays of multiple default processing units while using less than all of the memory cells of the plurality of processing units.

### Claim 16 (depends on 14)

The method of claim 14, wherein the method further comprises:
determining that the number of output maps generated for a particular one of the plurality of pixels in the particular layer exceeds a capacity of the default processing unit, and
configuring a corresponding one of the plurality of processing units to combine the memory cells of multiple default processing units while using less than all of the tensor arrays of the plurality of processing units.

### Claim 17

A controller comprising one or more processors configured to:
determine, for a particular layer of a neural network, a number of input channels and a number of output maps generated for particular ones of a plurality of pixels;
configure a portion of a processing chip into a plurality of processing units, wherein the processing chip includes one or more tensor arrays and one or more pixel arrays, wherein a particular processing unit performs computations associated with a particular one of the plurality of pixels; and wherein individual ones of the plurality of processing units include:
a number of the tensor arrays determined based on the number of input channels, and
a number of pixel arrays corresponding to the number of output maps; and
assign the plurality of processing units to perform computations of the particular layer.

### Claim 18 (depends on 17)

The controller of claim 17, wherein the one or more processors are further configured to:
provide input into the processing chip to perform the computations of the particular layer for individual ones of the plurality of pixels using the number of tensor arrays and to store output using the number of pixel arrays, and
provide an output of the processing chip as an output of the neural network.

### Claim 19 (depends on 17)

The controller of claim 17, wherein for a particular processing unit, the one or more processors are further configured to cause send, by the tensor arrays of the number of tensor arrays, output data to corresponding ones of the number of pixel arrays over a subset of the interconnects connecting particular ones of the tensor arrays to particular ones of the memory cells.

### Claim 20 (depends on 17)

The controller of claim 17, wherein a default processing unit includes one tensor array and one pixel array connected to one another by one interconnect connecting the one tensor array to the one memory cell for processing one channel and generate one output, and wherein the one or more processors are further configured to
determine that the number of input channels for a particular one of the plurality of pixels in the particular layer exceeds a capacity of the default processing unit, and
configure a corresponding one of the plurality of processing units to combine tensor arrays of multiple default processing units while using less than all of the pixel arrays of the plurality of processing units,
wherein the controller performs the operations by computer executable instructions stored in a non-transitory computer-readable medium.
