# US 2022/0036158 A1 · Task skew management for neural processor circuit

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0036158 A1
- Document ID: US20220036158A1
- Status: Published application
- Kind: A1
- Application number: 16/942,744
- Filed: 2020-07-29
- Published: 2022-02-03
- Earliest claimed: 2020-07-29
- Assignees: Apple Inc.
- Applicants: Apple Inc.
- Inventors: PONAN KUO
- Web page: https://patentgrep.com/patent/US20220036158A1

## CPC

- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/063: using electronic means
- G06N 3/08: Learning methods
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Embodiments relate to a neural processor circuit including one or more planar engine circuits that perform non-convolution operations in parallel with convolution operations performed by one or more neural engine circuits. The neural engine circuits perform the convolution operations on neural input data corresponding to one or more neural engine tasks to generate neural output data. The planar engine circuits perform non-convolution operations on planar input data corresponding to one or more planar engine tasks to generate planar output data. A data processor circuit that includes multiple buffer circuits performs task skew management between the one or more neural engine tasks and the one or more planar engine tasks. The data processor circuit stops addition of an incoming task to queues in response to one or more of the queues stored in the buffer circuits reaching a threshold.

## Claims

### Claim 1

A neural processor circuit comprising:
one or more neural engine circuits configured to perform convolution operations on neural input data corresponding to one or more neural engine tasks to generate neural output data;
one or more planar engine circuits configured to perform non-convolution operations on planar input data corresponding to one or more planar engine tasks to generate planar output data; and
a data processor circuit coupled to the one or more neural engine circuits and the one or more planar engine circuits, the data processor circuit comprising a plurality of buffer circuits for storing a plurality of queues of tasks, the plurality of buffer circuits comprising a first buffer for storing a first queue of the neural engine tasks for the one or more neural engine circuits to perform and a second buffer for storing a second queue of the planar engine tasks for the one or more planar engine circuits to perform, the data processor circuit configured to stop addition of an incoming task to the plurality of queues in response to one or more of the queues stored in the plurality of buffer circuits reaching a threshold.

### Claim 2 (depends on 1)

The neural processor circuit of claim 1, wherein the data processor circuit further comprises:
a first-in-first-out (FIFO) buffer circuit coupled to the plurality of buffer circuits, the FIFO buffer circuit upstream of the plurality of buffer circuits, the FIFO buffer circuit configured to receive the incoming task and hold the incoming task at the FIFO buffer circuit in response to the one or more of the queues stored in the plurality of buffer circuits reaches the threshold.

### Claim 3 (depends on 1)

The neural processor circuit of claim 1, wherein at least one of the queues corresponds to the tasks for a set of circuits, the set of circuits is the neural engine circuits, the planar engine circuits, or another set of circuits, one of the buffer circuits that stores the at least one of the queues comprises a filter that removes the tasks that are not intended for the set of circuits.

### Claim 4 (depends on 1)

The neural processor circuit of claim 1, wherein the data processor circuit further comprises a data control circuit configured to address data dependency between the neural engine tasks and the planar engine tasks.

### Claim 5 (depends on 1)

The neural processor circuit of claim 1, wherein at least one of the queues is stored in one of the buffer circuits as a linked list.

### Claim 6 (depends on 1)

The neural processor circuit of claim 1, further comprising a task manager circuit coupled to the data processor circuit, the task manager circuit configured to:
receive a task descriptor from a software compiling process, the task descriptor defining a configuration of the neural processor circuit to execute a corresponding neural engine task or a corresponding planar engine task, and
send a version of the task descriptor to the data processor circuit as the incoming task.

### Claim 7 (depends on 1)

The neural processor circuit of claim 1, wherein the data processor circuit further comprises a memory bank buffer coupled to the neural engine circuits and the planar engine circuits, the memory bank buffer configured to store the neural output data and the planar output data.

### Claim 8 (depends on 1)

The neural processor circuit of claim 1, wherein the threshold sets a maximum task skew between the neural engine tasks and the planar engine tasks.

### Claim 9 (depends on 1)

The neural processor circuit of claim 1, wherein the threshold is the same for the plurality of queues.

### Claim 10 (depends on 1)

The neural processor circuit of claim 1, wherein the threshold is programmable.

### Claim 11

A method for performing neural processing operations, comprising:
storing a plurality of queues of tasks in a plurality of buffer circuits comprising a first buffer and a second buffer, storing the plurality of queues comprising:
storing a first queue of neural engine tasks for one or more neural engine circuits to perform, and
storing a second queue of planar engine tasks for one or more planar engine circuits to perform;
performing, by the one or more neural engine circuits, convolution operations on neural input data corresponding to the neural engine tasks to generate neural output data;
performing, by the one or more planar engine circuits, non-convolution operations on planar input data corresponding to the planar engine tasks to generate planar output data; and
stopping addition of an incoming task to the plurality of queues in response to one or more of the queues stored in the plurality of buffer circuits reaching a threshold.

### Claim 12 (depends on 11)

The method of claim 11, wherein stopping addition of the incoming task to any of the queues comprises:
storing the incoming task in a first-in-first-out (FIFO) buffer circuit, the FIFO buffer circuit upstream of the plurality of buffer circuits;
holding the incoming task at the FIFO buffer circuit until the queues stored in the plurality of buffer circuits no longer reach the threshold.

### Claim 13 (depends on 11)

The method of claim 11, wherein at least one of the queues corresponds to the tasks for a set of circuits, the set of circuits is the neural engine circuits, the planar engine circuits, or another set of circuits, wherein the method further comprises:
filtering the tasks that are not intended for the set of circuits from the at least one of the queues.

### Claim 14 (depends on 11)

The method of claim 11, wherein at least one of the queues is stored in one of the buffer circuits as a linked list.

### Claim 15 (depends on 11)

The method of claim 11, further comprising:
receiving a task descriptor from a software compiling process, the task descriptor defining a configuration of the neural processor circuit to execute a corresponding neural engine task or a corresponding planar engine task, a version of the task descriptor corresponding to the incoming task.

### Claim 16 (depends on 11)

The method of claim 11, wherein the threshold sets a maximum task skew between the neural engine tasks and the planar engine tasks.

### Claim 17 (depends on 11)

The method of claim 11, wherein the threshold is programmable.

### Claim 18

An electronic device, comprising:
a memory storing a machine learning model; and
a neural processor, comprising:
one or more neural engine circuits configured to perform convolution operations on neural input data corresponding to one or more neural engine tasks to generate neural output data;
one or more planar engine circuits configured to perform non-convolution operations on planar input data corresponding to one or more planar engine tasks to generate planar output data; and
a data processor circuit coupled to the one or more neural engine circuits and the one or more planar engine circuits, the data processor circuit comprising a plurality of buffer circuits for storing a plurality of queues of tasks, the plurality of buffer circuits comprising a first buffer for storing a first queue of the neural engine tasks for the one or more neural engine circuits to perform and a second buffer for storing a second queue of the planar engine tasks for the one or more planar engine circuits to perform, the data processor circuit configured to stop addition of an incoming task compiled from the machine learning model to the plurality of queues in response to one or more of the queues stored in the plurality of buffer circuits reaching a threshold.

### Claim 19 (depends on 18)

The electronic device of claim 18, wherein the data processor circuit further comprises:
a first-in-first-out (FIFO) buffer circuit coupled to the plurality of buffer circuits, the FIFO buffer circuit upstream of the plurality of buffer circuits, the FIFO buffer circuit configured to receive the incoming task and hold the incoming task at the FIFO buffer circuit in response to the one or more of the queues stored in the plurality of buffer circuits reaches the threshold.

### Claim 20 (depends on 18)

The electronic device of claim 18, wherein at least one of the queues corresponds to the tasks for a set of circuits, the set of circuits is the neural engine circuits, the planar engine circuits, or another set of circuits, one of the buffer circuits that stores the at least one of the queues comprises a filter that removes the tasks that are not intended for the set of circuits.
