# US 2025/0232001 A1 · Low latency matrix multiply unit

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0232001 A1
- Document ID: US20250232001A1
- Status: Published application
- Kind: A1
- Application number: 19/023,004
- Filed: 2025-01-15
- Published: 2025-07-17
- Earliest claimed: 2017-05-17
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Andrew Everett Phelps; Norman Paul Jouppi
- Web page: https://patentgrep.com/patent/US20250232001A1

## CPC

- G06F 17/16: Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)}
- G06F 5/015: {having at least two separately controlled shifting levels, e.g. using shifting matrices (G06F5/012 takes precedence)}
- G06F 7/5443: {Sum of products (for applications thereof, see the relevant places, e.g. G06F17/10, H03H17/00)}
- G06F 9/30036: {Instructions to perform operations on packed data, e.g. vector, tile or matrix operations}
- G06F 9/30101: {Special purpose registers}
- G06F 15/8046: {Systolic arrays}
- G06N 3/063: using electronic means
- G06F 9/30032: {Movement instructions, e.g. MOVE, SHIFT, ROTATE, SHUFFLE}
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/08: Learning methods
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Methods, systems, and apparatus for a matrix multiply unit implemented as a systolic array of cells are disclosed. Each cell of the matrix multiply includes: a weight matrix register configured to receive a weight input from either a transposed or a non-transposed weight shift register; a transposed weight shift register configured to receive a weight input from a horizontal direction to be stored in the weight matrix register; a non-transposed weight shift register configured to receive a weight input from a vertical direction to be stored in the weight matrix register; and a multiply unit that is coupled to the weight matrix register and configured to multiply the weight input of the weight matrix register with a vector data input in order to obtain a multiplication result.

## Claims

### Claim 1

(canceled)

### Claim 2

A processor comprising:
a processing core having an array of cells;
a plurality of first shift chains, each first shift chain configured to shift weight values of a matrix;
a plurality of second shift chains, each second shift chain configured to shift transposed weight values of the matrix,
wherein the processor is configured to execute instructions that cause the processor to perform operations comprising:
loading each cell by selecting a weight value from a first shift chain of the plurality of first shift chains or selecting a transposed weight value from a second shift chain of the plurality of second shift chains, and
performing one or more matrix multiplication operations using the values loaded into the cells.

### Claim 3 (depends on 2)

The processor of claim 2, wherein the first plurality of shift chains corresponds to transposed weights and the second plurality of shift chains corresponds to non-transposed weights or wherein the first plurality of shift chains corresponds to non-transposed weights and the second plurality of shift chains corresponds to transposed weights.

### Claim 4 (depends on 2)

The processor of claim 2, wherein the operations further comprises loading the vector input using the first plurality of shift chains or the second plurality of shift chains.

### Claim 5 (depends on 2)

The processor of claim 2, further comprising a plurality of weight registers coupled to the first plurality of shift chains and the second plurality of shift chains.

### Claim 6 (depends on 2)

The processor of claim 2, wherein the processing core further includes one or more of a scalar memory, a vector memory, a scalar processing unit, one or more vector registers, a transposed unit, or a reduction and permutation unit.

### Claim 7

A method performed by a processor, wherein the processor comprises a processing core having an array of cells, a plurality of first shift chains, each first shift chain configured to shift weight values of a matrix, and a plurality of second shift chains, each second shift chain configured to shift transposed weight values of the matrix, the method comprising:
loading each cell by selecting a weight value from a first shift chain of the plurality of first shift chains or selecting a transposed weight value from a second shift chain of the plurality of second shift chains; and
performing one or more matrix multiplication operations using the values loaded into the cells.

### Claim 8 (depends on 7)

The method of claim 7, wherein the first plurality of shift chains corresponds to transposed weights and the second plurality of shift chains corresponds to non-transposed weights or wherein the first plurality of shift chains corresponds to non-transposed weights and the second plurality of shift chains corresponds to transposed weights.

### Claim 9 (depends on 7)

The method of claim 7, wherein the operations further comprises loading the vector input using the first plurality of shift chains or the second plurality of shift chains.

### Claim 10 (depends on 7)

The method of claim 7, further comprising a plurality of weight registers coupled to the first plurality of shift chains and the second plurality of shift chains.

### Claim 11 (depends on 7)

The method of claim 7, wherein the processing core further includes one or more of a scalar memory, a vector memory, a scalar processing unit, one or more vector registers, a transposed unit, or a reduction and permutation unit.

### Claim 12

A non-transitory computer program product storing instructions that, when executed by a programmable processor comprising a processing core having an array of cells, a plurality of first shift chains, each first shift chain configured to shift weight values of a matrix, and a plurality of second shift chains, each second shift chain configured to shift transposed weight values of the matrix, cause the processor to perform operations comprising:
loading each cell by selecting a weight value from a first shift chain of the plurality of first shift chains or selecting a transposed weight value from a second shift chain of the plurality of second shift chains; and
performing one or more matrix multiplication operations using the values loaded into the cells.

### Claim 13 (depends on 12)

The non-transitory computer program product of claim 12, wherein the first plurality of shift chains corresponds to transposed weights and the second plurality of shift chains corresponds to non-transposed weights or wherein the first plurality of shift chains corresponds to non-transposed weights and the second plurality of shift chains corresponds to transposed weights.

### Claim 14 (depends on 12)

The non-transitory computer program product of claim 12, wherein the operations further comprises loading the vector input using the first plurality of shift chains or the second plurality of shift chains.

### Claim 15 (depends on 12)

The non-transitory computer program product of claim 12, further comprising a plurality of weight registers coupled to the first plurality of shift chains and the second plurality of shift chains.

### Claim 16 (depends on 12)

The non-transitory computer program product of claim 12, wherein the processing core further includes one or more of a scalar memory, a vector memory, a scalar processing unit, one or more vector registers, a transposed unit, or a reduction and permutation unit.
