# US 2023/0244632 A1 · Neural processing accelerator

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0244632 A1
- Document ID: US20230244632A1
- Status: Published application
- Kind: A1
- Application number: 18/133,306
- Filed: 2023-04-11
- Published: 2023-08-03
- Earliest claimed: 2017-12-19
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Ilia OVSIANNIKOV; Yibing Michelle WANG
- Web page: https://patentgrep.com/patent/US20230244632A1

## CPC

- G06F 15/17343: {wherein the interconnection is dynamically configurable, e.g. having loosely coupled nearest neighbor architecture (reconfigurable processors arrays G06F15/7867)}
- G06F 7/5332: {by skipping over strings of zeroes or ones, e.g. using the Booth Algorithm}
- G06F 7/5443: {Sum of products (for applications thereof, see the relevant places, e.g. G06F17/10, H03H17/00)}
- G06F 7/57: Arithmetic logic units [ALU], i.e. arrangements or devices for performing two or more of the operations covered by groups G06F7/483 – G06F7/556 or for performing logical operations {(G06F7/49, G06F7/491 take precedence)}
- G06F 9/3889: {controlled by multiple instructions, e.g. MIMD, decoupled access or execute}
- G06F 9/3897: {with adaptable data path}
- G06F 13/1673: {using buffers}
- G06F 17/15: Correlation function computation {including computation of convolution operations (arithmetic circuits for sum of products per se, e.g. multiply-accumulators G06F7/5443; digital filters, e.g. FIR, IIR, adaptive filters H03H17/00)}
- G06F 17/16: Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)}
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/063: using electronic means
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A system for calculating. A scratch memory is connected to a plurality of configurable processing elements by a communication fabric including a plurality of configurable nodes. The scratch memory sends out a plurality of streams of data words. Each data word is either a configuration word used to set the configuration of a node or of a processing element, or a data word carrying an operand or a result of a calculation. Each processing element performs operations according to its current configuration and returns the results to the communication fabric, which conveys them back to the scratch memory.

## Claims

### Claim 1

A system for calculating, the system comprising:
a processing circuit comprising a memory device for storing a value,
the processing circuit being configured to:
receive, at a first input of the processing circuit, first data including a first portion and a second portion;
determine whether the first portion identifies a first type of command; and
in response to the first portion identifying the first type of command, store the second portion of the first data in the memory device.

### Claim 2 (depends on 1)

The system of claim 1, wherein the processing circuit is further configured to:
receive, at a second input of the processing circuit, second data including a third portion and a fourth portion;
determine whether the first portion and the third portion identify the first type of command; and
in response to neither the first portion nor the third portion identifying the first type of command, perform an operation on the first data and the second data, in accordance with the value stored in the memory device.

### Claim 3 (depends on 2)

The system of claim 2, wherein the performing of the operation comprises:
in response to the value indicating a multiplication operation, multiplying the second portion of the first data by the fourth portion of the second data.

### Claim 4 (depends on 2)

The system of claim 2, wherein the performing of the operation comprises:
in response to the value indicating an addition operation, adding the second portion of the first data with the fourth portion of the second data.

### Claim 5 (depends on 2)

The system of claim 2, wherein the performing of the operation comprises:
in response to the value indicating a multiplication and accumulation:
forming a product by multiplying the second portion of the first data by the fourth portion of the second data, and
adding the product to an accumulator value.

### Claim 6 (depends on 1)

The system of claim 1, wherein the processing circuit includes 5,000 or fewer gates.

### Claim 7

A system for calculating, the system comprising:
a communication fabric comprising a plurality of first circuits, at least one of the plurality of first circuits comprising at least a second circuit, the second circuit comprising:
a first input configured to receive first data having a first portion and a second portion;
a first output;
a first memory device; and
a second memory device configured to store a value,
wherein, in response to the first portion indicating a first command, the second circuit is configured to store the second portion in the second memory device.

### Claim 8 (depends on 7)

The system of claim 7, wherein the at least the second circuit includes:
a second input configured to receive second data having a third portion and a fourth portion, wherein in response to neither the second portion nor the third portion identifying the first command, the second circuit is configured to, based on the value in the second memory device, perform at least one of: sending the first data to the first memory device,
sending the first data to the first output; or
sending the first data from the first memory device to the first output.

### Claim 9 (depends on 7)

The system of claim 7, wherein at least one of the plurality of first circuits includes:
four second inputs extending outward from the at least one of the plurality of first circuits in substantially orthogonal directions and
four second outputs extending outward from the at least one of the plurality of first circuits in substantially orthogonal directions.

### Claim 10 (depends on 9)

The system of claim 9, wherein the at least one of the plurality of first circuits comprises four of the second circuits, each of the second circuits having:
four of the first inputs connected respectively to the four second inputs of the at least one of the plurality of first circuits, wherein the first output is connected to a respective one of the four second outputs of the at least one of the plurality of first circuits.

### Claim 11 (depends on 7)

The system of claim 7, wherein the at least one of the plurality of first circuits includes 2,000 or fewer gates.

### Claim 12 (depends on 7)

The system of claim 7, further comprising a third memory comprising:
a memory bank; and
a memory controller,
the memory controller comprising a plurality of row caches, wherein at least one of the plurality of row caches has a same number of bits as the first data, the memory controller being configured to transfer second data to the row caches from the memory bank, and transfer third data to the memory bank from the row caches,
one or more of the row caches being configured to:
stream out a sequence of the second data, and
stream in a sequence of the third data.

### Claim 13 (depends on 12)

The system of claim 12, wherein one of the row caches includes a double buffer.

### Claim 14 (depends on 12)

The system of claim 12, wherein:
the first portion of the second data has a width of four bits, and
the second portion of the second data has a width of sixteen bits.

### Claim 15

A method for calculating via a system comprising a first circuit comprising a plurality of second circuits, each of the second circuits having a plurality of first inputs, a first output, a first memory device, and a second memory device storing a value, the method comprising:
receiving, via the plurality of first inputs of at least one of the second circuits, a first plurality of data, each of the first plurality of data being received via at least one of the plurality of first inputs and having a first portion and a second portion;
determining whether one of the first plurality of data has a first portion identifying a first type of command; and
in response to determining that the one of the first plurality of data has the first portion identifying the first type of command, storing the second portion of the one of the first plurality of data in the second memory device.

### Claim 16 (depends on 15)

The method of claim 15, further comprising:
receiving, via the plurality of first inputs of the at least one of the second circuits, a second plurality of data, each of the second plurality of data being received via at least one of the plurality of first inputs and having a third portion and a fourth portion;
determining whether the third portion of the second plurality of data identify the first type of command; and
in response to determining that none of the second plurality of data identify the first type of command, performing, based on the value in the second memory device, at least one of:
sending one of the second plurality of data from one of the plurality of inputs to the first memory device,
sending one of the second plurality of data from one of the plurality of inputs to the output of the at least one of the plurality of second circuits, or
sending one of the second plurality of data from the first memory to the output of the at least one of the plurality of second circuits.

### Claim 17 (depends on 15)

The method of claim 15, wherein the first circuit has four second inputs and four second outputs, and comprises four of the second circuits, each of the second circuits having:
four of the first inputs connected respectively to the four second inputs of the first circuit, wherein the first output is connected to at least one of the four second outputs of the first circuit.

### Claim 18 (depends on 15)

The method of claim 15, wherein the first circuit includes 2,000 or fewer gates.

### Claim 19 (depends on 15)

The method of claim 15, wherein the system for calculating further comprises a third memory comprising:
a memory bank; and
a memory controller,
the memory controller comprising a plurality of row caches, wherein at least one of the plurality of row caches has a same number of bits as the first data, the memory controller being configured to transfer second data to the row caches from the memory bank, and transfer third data to the memory bank from the row caches,
one or more of the row caches being configured to:
stream out a sequence of the second data, and
stream in a sequence of the third data.

### Claim 20 (depends on 19)

The method of claim 19, wherein one of the row caches includes a double buffer.
