# US 2025/0165763 A1 · Dynamically shaping and segmenting work units for processing in neural network processor

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0165763 A1
- Document ID: US20250165763A1
- Status: Published application
- Kind: A1
- Application number: 19/030,985
- Filed: 2025-01-17
- Published: 2025-05-22
- Earliest claimed: 2018-05-04
- Assignees: Apple Inc.
- Applicants: Apple Inc
- Inventors: Christopher L. Mills
- Web page: https://patentgrep.com/patent/US20250165763A1

## CPC

- G06N 3/063: using electronic means
- G06F 13/1673: {using buffers}
- G06F 17/15: Correlation function computation {including computation of convolution operations (arithmetic circuits for sum of products per se, e.g. multiply-accumulators G06F7/5443; digital filters, e.g. FIR, IIR, adaptive filters H03H17/00)}
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Embodiments relate to a neural processor circuit that includes multiple neural engine circuits, a data buffer, and a kernel fetcher circuit. At least one of the neural engine circuits receives multiple sub-channels of a portion of input data from the data buffer. Neural engine circuit further receives a kernel of the one or more kernels from the kernel fetcher circuit, wherein the kernel was decomposed into a corresponding sub-kernel for each sub-channel of the portion of the input data. Neural engine circuit performs a convolution operation on each sub-channel of the portion of the input data and the corresponding sub-kernel. Neural engine circuit accumulates corresponding outputs of each sub-channel portion of the convolution operation to generate a single channel of the output data.

## Claims

### Claim 1

A neural processor circuit, comprising:
a neural engine circuit comprising an input buffer and an operation unit;
a data buffer;
a data reader comprising a first rasterizer circuit configured to cause the data reader to receive at least a first portion of input data from a system memory external to the neural processor circuit and to store the first portion of input data in the data buffer, wherein the data buffer comprises a second rasterizer circuit configured to cause the data buffer to send a second portion of input data selected from the first portion of input data to the neural engine circuit; and
a kernel fetcher circuit comprising a third rasterizer circuit configured to cause the kernel fetcher circuit to receive a kernel from the system memory and to send the kernel to the neural engine circuit, wherein the neural engine circuit comprises a fourth rasterizer circuit configured to store the second portion of input data in the input buffer of the neural engine circuit and to forward a third portion of input data selected from the second portion of input data to the operation unit of the neural engine circuit for processing to generate output data based on the third portion of input data and the kernel.

### Claim 2 (depends on 1)

The neural processor circuit of claim 1, wherein the second portion of input data is a part of the first portion of input data and the third portion of input data is a part of the second portion of input data.

### Claim 3 (depends on 1)

The neural processor circuit of claim 1, wherein the first portion of input data comprises a tile of input data, the second portion of input data comprises a work unit of the tile of input data, and the third portion of input data is a part of the work unit.

### Claim 4 (depends on 1)

The neural processor circuit of claim 1, wherein the operation unit of the neural engine circuit comprises a multiply-accumulator including an accumulator and a multiply-add circuit.

### Claim 5 (depends on 1)

The neural processor circuit of claim 1, wherein the fourth rasterizer circuit of the neural engine circuit is further configured to send the output data to the data buffer.

### Claim 6 (depends on 1)

The neural processor circuit of claim 1, wherein the kernel fetcher circuit is shared by the neural engine circuit and one or more other neural engine circuits.

### Claim 7 (depends on 1)

The neural processor circuit of claim 1, wherein the neural processor circuit further comprises a task manager configured to send task information to the first rasterizer circuit, the second rasterizer circuit, the third rasterizer circuit, and the fourth rasterizer circuit.

### Claim 8 (depends on 1)

The neural processor circuit of claim 1, wherein the neural engine circuit is further configured to:
determine whether the second portion of input data stored in the input buffer has been entirely processed; and
in response determining that the second portion of input data has not been entirely processed, forward a fourth portion of input data selected from the second portion of input data to the operation unit of the neural engine circuit for processing to generate another output data based on the fourth portion of input data.

### Claim 9 (depends on 8)

The neural processor circuit of claim 8, wherein the fourth portion of input data and the third portion of input data are a part of the second portion of input data, and wherein the fourth portion of input data is generated by shifting the second portion of input data stored in the input buffer.

### Claim 10 (depends on 1)

The neural processor circuit of claim 1, wherein:
the neural engine circuit is further configured to determine whether the second portion of input data stored in the input buffer has been entirely processed; and
in response determining that the second portion of input data has been entirely processed, the second rasterizer circuit is further configured to cause the data buffer to send a fourth portion of input data selected from the first portion of input data to the neural engine circuit for processing.

### Claim 11 (depends on 10)

The neural processor circuit of claim 10, wherein the first portion of input data comprises a tile of input data, the second portion of input data comprises a first work unit of the tile of input data, and the fourth portion of input data comprises a second work unit of the tile of input data.

### Claim 12

A method for processing input data by a neural processor circuit, comprising:
causing, by a first rasterizer circuit of a data reader, the data reader to receive a first portion of input data from a system memory external to the neural processor circuit;
storing the first portion of input data in a data buffer;
causing, by a second rasterizer circuit of the data buffer, the data buffer to send a second portion of input data selected from the first portion of input data to a neural engine circuit the neural processor circuit;
causing, by a third rasterizer circuit of a kernel fetcher circuit, the kernel fetcher circuit to receive a kernel from the system memory and to send the kernel to the neural engine circuit;
storing, by a fourth rasterizer circuit of the neural engine circuit, the second portion of input data in an input buffer of the neural engine circuit; and
forwarding a third portion of input data selected from the second portion of input data to an operation unit of the neural engine circuit for processing to generate output data based on the third portion of input data and the kernel.

### Claim 13 (depends on 12)

The method of claim 12, wherein the first portion of input data comprises a tile of input data, the second portion of input data comprises a work unit of the tile of input data, and the third portion of input data is a part of the work unit.

### Claim 14 (depends on 12)

The method of claim 12, wherein the operation unit of the neural engine circuit comprises a multiply-accumulator including an accumulator and a multiply-add circuit.

### Claim 15 (depends on 12)

The method of claim 12, further comprising:
sending the output data to the data buffer.

### Claim 16 (depends on 12)

The method of claim 12, wherein the kernel fetcher circuit is shared by the neural engine circuit and one or more other neural engine circuits.

### Claim 17

A system, comprising:
a system memory configured to store input data; and
a neural processor circuit external to the system memory, comprising:
a neural engine circuit;
a data buffer;
a data reader comprising a first rasterizer circuit configured to cause the data reader to receive at least a first portion of input data from the system memory and to store the first portion of input data in the data buffer, wherein:
the data buffer comprises a second rasterizer circuit configured to cause the data buffer to send a second portion of input data selected from the first portion of input data to the neural engine circuit; and
the neural engine circuit comprises a third rasterizer circuit, an input buffer, and an operation unit, wherein the third rasterizer circuit is configured to store the second portion of input data in the input buffer and to forward a third portion of input data selected from the second portion of input data to the operation unit for processing to generate output data based on the third portion of input data.

### Claim 18 (depends on 17)

The system of claim 17, wherein the neural processor circuit further comprises a kernel fetcher circuit comprising a fourth rasterizer circuit configured to cause the kernel fetcher circuit to receive a kernel from the system memory and to send the kernel to the neural engine circuit, and wherein the neural engine circuit is configured to generate the output data based on the third portion of input data and the kernel.

### Claim 19 (depends on 17)

The system of claim 17, wherein the first portion of input data comprises a tile of input data, the second portion of input data comprises a work unit of the tile of input data, and the third portion of input data comprises a part of the work unit.

### Claim 20 (depends on 17)

The system of claim 17, wherein the third rasterizer circuit of the neural engine circuit is further configured to send the output data to the data buffer.
