# US 12,217,160 B1 · Allocating blocks of unified memory for integrated circuit executing neural network

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,217,160 B1
- Document ID: US12217160B1
- Status: Granted
- Kind: B1
- Application number: 17/306,745
- Filed: 2021-05-03
- Granted: 2025-02-04
- Earliest claimed: 2021-04-23
- Assignees: Amazon Technologies, Inc.
- Applicants: Perceive Corporation
- Inventors: Jung Ko; Kenneth Duong; Steven L. Teig; Won Rhee
- Web page: https://patentgrep.com/patent/US12217160B1

## CPC

- G06N 3/045: Combinations of networks
- G06F 9/223: {Execution means for microinstructions irrespective of the microinstruction function, e.g. decoding of microinstructions and nanoinstructions; timing of microinstructions; programmable logic arrays; delays and fan-out problems}
- G06F 12/0246: {in block erasable memory, e.g. flash memory}
- G06F 12/06: Addressing a physical block of locations, e.g. base addressing, module addressing, memory dedication (G06F12/08 takes precedence)
- G06F 12/0802: Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
- G06F 12/10: Address translation
- G06N 3/048: Activation functions
- G06N 3/063: using electronic means
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Some embodiments provide a method that receives a specification of a neural network for execution by an integrated circuit. The integrated circuit includes a neural network inference circuit for executing the neural network to generate an output based on an input, an input processing circuit for providing the input to the neural network inference circuit, a microprocessor circuit for controlling the neural network inference circuit and the input processing circuit, and a unified memory accessible by the microprocessor circuit, the neural network inference circuit, and the input processing circuit. The method determines usage of the unified memory by the neural network inference circuit while executing the neural network. Based on the determined usage by the neural network inference circuit, the method allocates portions of the unified memory to the microprocessor circuit and input processing circuit.

## Claims

### Claim 1

A method for a compiler that generates configuration data to enable an integrated circuit (IC) to execute a neural network, the method comprising:
receiving a specification of the neural network for execution by the integrated circuit, the integrated circuit comprising (i) a neural network inference circuit for executing the neural network to generate an output based on an input, (ii) an input processing circuit for providing the input to the neural network inference circuit, (iii) a microprocessor circuit for controlling the neural network inference circuit and the input processing circuit, and (iv) a unified memory accessible by the microprocessor circuit, the neural network inference circuit, and the input processing circuit;
determining portions of the unified memory for the neural network inference circuit to use when the neural network inference circuit executes the neural network; and
based on the determined portions of the unified memory that the neural network inference circuit will use when executing the neural network, allocating other portions of the unified memory to the microprocessor circuit and input processing circuit.

### Claim 2 (depends on 1)

The method of claim 1, wherein the neural network inference circuit stores weight values and activation values in the unified memory.

### Claim 3 (depends on 2)

The method of claim 2, wherein the amount of unified memory required by the neural network inference circuit varies during execution of the neural network based on a number of activation values stored in the unified memory.

### Claim 4 (depends on 1)

The method of claim 1, wherein the integrated circuit (i) is incorporated into a device that comprises a sensor, (ii) receives inputs from the sensor, and (iii) stores the inputs in the unified memory in order for the neural network inference circuit to generate outputs based on the inputs according to instructions from the input processing circuit.

### Claim 5 (depends on 1)

The method of claim 1, wherein the unified memory is organized as a plurality of memory banks.

### Claim 6 (depends on 5)

The method of claim 5, wherein determining portions of the unified memory for the neural network inference circuit to use comprises generating neural network program instructions that specify memory banks used by the neural network inference circuit for storing weight values and activation values at each of a plurality of layers of the neural network.

### Claim 7 (depends on 6)

The method of claim 6, wherein allocating portions of the unified memory to the microprocessor circuit and input processing circuit comprises:
based on a size of the inputs to be provided to the neural network inference circuit and a number of the inputs to be stored in the unified memory at the same time, allocating memory banks to the input processing circuit for storing the inputs; and
allocating a set of memory banks to the microprocessor circuit that are not already allocated to the neural network inference circuit or the input processing circuit.

### Claim 8 (depends on 5)

The method of claim 5, wherein:
the neural network inference circuit comprises a plurality of cores;
each respective core is associated with a respective set of the memory banks of the unified memory such that each memory bank is associated with one of the cores and no memory bank is associated with multiple cores; and
different cores use different numbers of memory banks when the neural network inference circuit executes the neural network.

### Claim 9 (depends on 8)

The method of claim 8, wherein the memory banks that are not used by the cores of the neural network inference circuit are organized into a single set of memory banks for a virtual addressing scheme.

### Claim 10 (depends on 9)

The method of claim 9, wherein the virtual addressing scheme is used by the microprocessor circuit and the input processing circuit when addressing the unified memory.

### Claim 11

A non-transitory machine-readable medium storing a program for execution by at least one processing unit, the program comprising sets of instructions for:
receiving a specification of a neural network for execution by an integrated circuit that comprises a neural network inference circuit for executing the neural network to generate an output based on an input, an input processing circuit for providing the input to the neural network inference circuit, a microprocessor circuit for controlling the neural network inference circuit and the input processing circuit, and a unified memory accessible by the microprocessor circuit, the neural network inference circuit, and the input processing circuit;
determining usage of the unified memory by the neural network inference circuit while executing the neural network; and
based on the determined usage by the neural network inference circuit, allocating portions of the unified memory to the microprocessor circuit and input processing circuit.

### Claim 12 (depends on 11)

The non-transitory machine-readable medium of claim 11, wherein the neural network inference circuit stores weight values and activation values in the unified memory.

### Claim 13 (depends on 12)

The non-transitory machine-readable medium of claim 12, wherein the amount of unified memory required by the neural network inference circuit varies during execution of the neural network based on a number of activation values stored in the unified memory.

### Claim 14 (depends on 11)

The non-transitory machine-readable medium of claim 11, wherein the integrated circuit (i) is incorporated into a device that comprises a sensor, (ii) receives inputs from the sensor, and (iii) stores the inputs in the unified memory in order for the neural network inference circuit to generate outputs based on the inputs according to instructions from the input processing circuit.

### Claim 15 (depends on 11)

The non-transitory machine-readable medium of claim 11, wherein the unified memory is organized as a plurality of memory banks.

### Claim 16 (depends on 15)

The non-transitory machine-readable medium of claim 15, wherein the set of instructions for determining usage of the unified memory by the neural network inference circuit comprises a set of instructions for generating neural network program instructions that specify memory banks used by the neural network inference circuit for storing weight values and activation values at each of a plurality of layers of the neural network.

### Claim 17 (depends on 16)

The non-transitory machine-readable medium of claim 16, wherein the set of instructions for allocating portions of the unified memory to the microprocessor circuit and input processing circuit comprises sets of instructions for:
based on a size of the inputs to be provided to the neural network inference circuit and a number of the inputs to be stored in the unified memory at a given time, allocating memory banks to the input processing circuit for storing the inputs; and
allocating a set of memory banks to the microprocessor circuit that are not already allocated to the neural network inference circuit or the input processing circuit.

### Claim 18 (depends on 15)

The non-transitory machine-readable medium of claim 15, wherein:
the neural network inference circuit comprises a plurality of cores;
each respective core is associated with a respective set of the memory banks of the unified memory such that each memory bank is associated with one of the cores and no memory bank is associated with multiple cores; and
different cores use different numbers of memory banks when the neural network inference circuit executes the neural network.

### Claim 19 (depends on 18)

The non-transitory machine-readable medium of claim 18, wherein the memory banks that are not used by the cores of the neural network inference circuit are organized into a single set of memory banks for a virtual addressing scheme.

### Claim 20 (depends on 19)

The non-transitory machine-readable medium of claim 19, wherein the virtual addressing scheme is used by the microprocessor circuit and the input processing circuit when addressing the unified memory.
