# US 12,118,060 B2 · Computational circuit with hierarchical accumulator

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,118,060 B2
- Document ID: US12118060B2
- Status: Granted
- Kind: B2
- Application number: 17/545,860
- Filed: 2021-12-08
- Published: 2023-06-08
- Granted: 2024-10-15
- Earliest claimed: 2021-12-08
- Assignees: Tenstorrent Inc.
- Applicants: Tenstorrent Inc.
- Inventors: Davor Capalija; Ljubisa Bajic; Alex Cejkov
- Web page: https://patentgrep.com/patent/US12118060B2

## CPC

- G06F 7/509: for multiple operands, e.g. digital integrators
- G06F 17/16: Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)}
- G06F 7/5443: {Sum of products (for applications thereof, see the relevant places, e.g. G06F17/10, H03H17/00)}
- G11C 11/419: Read-write [R-W] circuits
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G11C: STATIC STORES (semiconductor memory devices H10B)

## Abstract

Methods and systems relating to computational circuitry are disclosed herein. A disclosed computational circuit includes a math circuit, a first accumulator, and a second accumulator. The first accumulator has a first memory. The second accumulator has a second memory. The first accumulator is communicatively connected to the math circuit and accumulates values from the math circuit in the first memory. The second accumulator is communicatively connected to the first memory and accumulates values from the first memory in the second memory. The first memory is faster and smaller than the second memory.

## Claims

### Claim 1

A computational circuit comprising:
a math circuit;
a first accumulator communicatively connected to the math circuit, having a first memory, and that accumulates values from the math circuit in the first memory;
a second accumulator communicatively connected to the first memory, having a second memory, and that accumulates values from the first memory in the second memory; and
wherein the first memory is faster and smaller than the second memory.

### Claim 2 (depends on 1)

The computational circuit of claim 1, further comprising:
a first sector of the first memory;
a second sector of the first memory; and
wherein: (i) the first accumulator accumulates in the first sector of the first memory when the second accumulator accumulates from the second sector of the first memory; and (ii) the first accumulator accumulates in the second sector of the first memory when the second accumulator accumulates from the first sector of the first memory.

### Claim 3 (depends on 2)

The computational circuit of claim 2, wherein:
the math circuit generates an output with an output size; and
the first sector of the first memory is at least as large as the output size.

### Claim 4 (depends on 3)

The computational circuit of claim 3, wherein:
the first sector of the first memory is equal in size to the output size; and
the second sector of the first memory is equal in size to the output size.

### Claim 5 (depends on 1)

The computational circuit of claim 1, wherein:
the math circuit outputs values at an output speed;
the first accumulator accumulates a value from the math circuit in the first memory at a first accumulation operation speed;
the first accumulation operation speed is at least as fast as the output speed;
the second accumulator accumulates a value from the first memory in the second memory at a second accumulation operation speed; and
the second accumulation operation speed is slower than the first accumulation operation speed.

### Claim 6 (depends on 5)

The computational circuit of claim 5, wherein:
the first accumulator conducts a number of accumulation operations to store an output data value in the first memory in response to a set of operands being applied to the math circuit;
the second accumulator conducts a number of accumulation operation to store the output data value in the second memory in response to the set of operands being applied to the math circuit; and
the number of accumulation operations to store the output data value in the first memory is greater than the number of accumulation operation to store the output data value in the second memory.

### Claim 7 (depends on 1)

The computational circuit of claim 1, wherein:
the math circuit is a multiplier array;
the math circuit generates a set of products required to compute an output matrix; and
the first memory is at least as large as the output matrix.

### Claim 8 (depends on 1)

A system using the computational circuit of claim 1, further comprising:
a controller;
wherein the computational circuit is a matrix multiplier circuit;
wherein the controller is programmed to: (i) multiply a first matrix and a second matrix using the matrix multiplier circuit to generate an output matrix; (ii) provide the first matrix to the matrix multiplier circuit in a first series of blocks; and (iii) provide the second matrix to the matrix multiplier circuit in a second series of blocks; and
wherein the second memory is at least as large as the output matrix.

### Claim 9 (depends on 8)

The system of claim 8, further comprising:
a network;
a remote memory that is accessible via the network;
a local memory located on a substrate with the computational circuit;
wherein the first series of blocks are provided from the local memory; and
wherein the second series of blocks are provided from the remote memory.

### Claim 10 (depends on 1)

The computational circuit of claim 1, wherein:
the first memory is a register memory; and
the second memory is a static random-access memory.

### Claim 11

A method comprising:
accumulating, using a first accumulator with a first memory, values from a math circuit in the first memory;
accumulating, using a second accumulator with a second memory, values from the first memory in the second memory; and
wherein the first memory is faster and smaller than the second memory.

### Claim 12 (depends on 11)

The method of claim 11, further comprising:
accumulating, using the first accumulator with the first memory, values from the math circuit in a first sector of the first memory while accumulating, using the second accumulator with the second memory, values from a second sector of the first memory in the second memory; and
accumulating, using the first accumulator with the first memory, values from the math circuit in the second sector of the first memory while accumulating, using the second accumulator with the second memory, values from the first sector of the first memory in the second memory.

### Claim 13 (depends on 12)

The method of claim 12, further comprising:
generating, using the math circuit, an output with an output size; and
wherein the first sector of the first memory is at least as large as the output size.

### Claim 14 (depends on 13)

The method of claim 13, wherein:
the first sector of the first memory is equal in size to the output size; and
the second sector of the first memory is equal in size to the output size.

### Claim 15 (depends on 12)

The method of claim 12, further comprising:
outputting values for the first accumulator using the math circuit and at an output speed;
wherein the accumulating, using the first accumulator, is done at a first per value accumulation speed which is at least as fast as the output speed; and
wherein the accumulating, using the second accumulator, is at a second per value accumulation speed which is slower than first per value accumulation speed.

### Claim 16 (depends on 15)

The method of claim 15, wherein:
the first accumulator conducts a number of accumulation operations to store an output data value in the first memory in response to a set of operands being applied to the math circuit;
the second accumulator conducts a number of accumulation operation to store the output data value in the second memory in response to the set of operands being applied to the math circuit; and
the number of accumulation operations to store the output data value in the first memory is greater than the number of accumulation operation to store the output data value in the second memory.

### Claim 17 (depends on 11)

The method of claim 11, wherein:
the math circuit is a multiplier array;
the math circuit generates a set of products required to compute an output; and
the first memory is at least twice as large as the set of products.

### Claim 18 (depends on 11)

The method of claim 11, further comprising:
multiplying a first matrix and a second matrix to produce an output matrix;
providing the first matrix and the second matrix to a computational circuit including the math circuit as a set of blocks;
accepting, using the computational circuit, the set of blocks as operands;
wherein the values from the math circuit are generated using the set of blocks as operands; and
wherein the second memory is at least as large as the output matrix.

### Claim 19 (depends on 11)

The method of claim 11, wherein:
the first memory is a register memory; and
the second memory is a static random-access memory.