# US 2022/0036163 A1 · Chained neural engine write-back architecture

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0036163 A1
- Document ID: US20220036163A1
- Status: Published application
- Kind: A1
- Application number: 16/942,263
- Filed: 2020-07-29
- Published: 2022-02-03
- Earliest claimed: 2020-07-29
- Assignees: Apple Inc.
- Applicants: Apple Inc.
- Inventors: Ponan Kuo; Hsiao-Chen Chang; Ji Liang Song
- Web page: https://patentgrep.com/patent/US20220036163A1

## CPC

- G06N 3/063: using electronic means
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Embodiments relate to a neural processor circuit that includes a first number of neural engine circuits, a second number of channels and a data processor circuit. The first number of neural engine circuits are pipelined into the second number of chains smaller than the first number. Each of the chains is configured to generate output data of a first size. Each of the channels is coupled to each of the chains and configured to transmit the output data from each of the neural engine circuits in the chains sequentially. The data processor circuit is coupled to the channels to receive the output data. The data processor circuit aggregates the output data of each of the chains into aggregated data of a second size larger than the first size and writes the aggregated data of the second size into a buffer memory of the data processor circuit.

## Claims

### Claim 1

A neural processor circuit, comprising:
a first number of neural engine circuits pipelined into a second number of chains smaller than the first number, each of the chains configured to generate output data of a first size;
the second number of channels, each of the channels coupled to each of the chains and configured to transmit the output data from each of the neural engine circuits in the chains sequentially; and
a data processor circuit coupled to the channels to receive the output data, the data processor circuit configured to:
aggregate the output data of each of the chains into aggregated data of a second size larger than the first size, and
write the aggregated data of the second size into a buffer memory of the data processor circuit.

### Claim 2 (depends on 1)

The neural processor circuit of claim 1, wherein each of the chains comprises a first neural engine circuit, a second neural engine circuit and a third neural engine circuit, the second neural engine circuit configured to:
generate first data of the first size, and
send the first data to the third neural engine circuit or forward second data of the first size generated by the first neural engine circuit to the third neural engine circuit, the aggregated data comprising the first data and the second data.

### Claim 3 (depends on 1)

The neural processor circuit of claim 1, wherein the data processor circuit comprises:
the second number of chain buffers configured to store the output data from the chains;
internal channels coupled to the chain buffers to carry the output data in units of the second size, and
a flow control circuit configured to select the aggregated data for writing from the chain buffers into the buffer memory.

### Claim 4 (depends on 3)

The neural processor circuit of claim 3, wherein the flow control circuit is further configured to generate a control signal for each of the chains to control propagating data through each of the chains.

### Claim 5 (depends on 1)

The neural processor circuit of claim 1, wherein each of the chains comprises a first neural engine circuit configured to:
forward, based on a flow control signal, first data of the first size generated by a second neural engine circuit in each of the chains to a third neural engine circuit in each of the chains, or
forward, based on the control signal, second data of the first size generated by the first neural engine circuit to the third neural engine circuit, the aggregated data comprising the first data and the second data.

### Claim 6 (depends on 1)

The neural processor circuit of claim 1, further comprising a flow control circuit configured to generate a control signal for each of the chains to output the output data from each of the chains into the second number of chain buffers of the data processor circuit.

### Claim 7 (depends on 1)

The neural processor circuit of claim 1, wherein each neural engine circuit in each of the chains includes a neural engine buffer configured to store first data of the first size generated by another neural engine circuit in each of the chains, and the aggregated data comprising the first data.

### Claim 8 (depends on 1)

The neural processor circuit of claim 1, wherein each of the chains comprises a first neural engine circuit including a pack forward circuit configured to forward first data of the first size into a neural engine buffer of a second neural engine circuit in each of the chains, and the aggregated data comprising the first data.

### Claim 9 (depends on 8)

The neural processor circuit of claim 8, wherein the first data was generated by the first neural engine circuit or the first data was received in a neural engine buffer of the first neural engine circuit from a third neural engine circuit in each of the chains.

### Claim 10

A method comprising:
generating, by each chain of a first number of chains, output data of a first size, a second number of neural engine circuits pipelined into the first number of chains smaller than the second number;
sending the results data from each of the neural engine circuits in the chains to chain buffers sequentially via channels coupled to the chains responsive to generating the output data;
aggregating the output data of each of the chains into aggregated data of a second size larger than the first size; and
writing the aggregated data of the second size into a buffer memory.

### Claim 11 (depends on 10)

The method of claim 10, further comprising:
generating, by a first neural engine circuit in each of the chains, first data of the first size; and
sending, by the first neural engine circuit, the first data to a second neural engine circuit in each of the chains; or
forwarding, by the first neural engine circuit, second data of the first size generated by a third neural engine circuit in each of the chains to the second neural engine circuit, the aggregated data comprising the first data and the second data.

### Claim 12 (depends on 10)

The method of claim 10, further comprising:
storing the output data from the chains into the first number of chain buffers;
carrying the output data in units of the second size via internal channels coupled to the chain buffers; and
selecting the aggregated data for writing from the chain buffers into the buffer memory.

### Claim 13 (depends on 10)

The method of claim 10, further comprising generating a control signal for each of the chains to control propagating data through each of the chains.

### Claim 14 (depends on 10)

The method of claim 10, further comprising:
forwarding, by a first neural engine circuit in each of the chains based on a flow control signal, first data of the first size generated by a second neural engine circuit in each of the chains to a third neural engine circuit in each of the chains, or
forwarding, based on the flow control signal, second data of the first size generated by the first neural engine circuit to the third neural engine circuit, the aggregated data comprising the first data and the second data.

### Claim 15 (depends on 10)

The method of claim 10, further comprising generating a control signal for each of the chains to output the output data from each of the chains into the first number of chain buffers.

### Claim 16 (depends on 10)

The method of claim 10, further comprising storing, in neural engine buffer of each neural engine circuit in each of the chains, first data of the first size generated by another neural engine circuit in each of the chains, the aggregated data comprising the first data.

### Claim 17 (depends on 10)

The method of claim 10, further comprising forwarding, from a first neural engine circuit in each of the chains, first data of the first size into a neural engine buffer of a second neural engine circuit in each of the chains, the aggregated data comprising the first data.

### Claim 18 (depends on 17)

The method of claim 17, further comprising:
generating the first data by the first neural engine circuit, or
receiving the first data in a neural engine buffer of the first neural engine circuit, the first data generated by a third neural engine circuit in each of the chains, the aggregated data comprising the first data.

### Claim 19

An electronic device, comprising:
a neural processor circuit including:
a first number of neural engine circuits pipelined into a second number of chains smaller than the first number, each of the chains configured to generate output data of a first size;
the second number of channels, each of the channels coupled to each of the chains and configured to transmit the output data from each of the neural engine circuits in the chains sequentially;
a data processor circuit coupled to the channels to receive the output data, the data processor circuit configured to:
aggregate the output data of each of the chains into aggregated data of a second size larger than the first size, and
write the aggregated data of the second size into a buffer memory of the data processor circuit; and
a planar engine circuit coupled to the data processor circuit, the planar engine circuit configured to:
receive data of the second size from the buffer memory, and
perform at least one operation on the received data of the second size.

### Claim 20 (depends on 19)

The electronic device of claim 19, wherein each of the chains comprises a first neural engine circuit, a second neural engine circuit and a third neural engine circuit, the second neural engine circuit configured to:
generate first data of the first size, and
send the first data to the third neural engine circuit or forward second data of the first size generated by the first neural engine circuit to the third neural engine circuit, the aggregated data comprising the first data and the second data.
