US 11,907,330 B2Granted

Low latency matrix multiply unit

AssigneeGoogle Inc.

InventorsAndrew Everett Phelps, Norman Paul Jouppi

Life of this patent

Estimating the term…

Loading patent…

Abstract

Methods, systems, and apparatus for a matrix multiply unit implemented as a systolic array of cells are disclosed. Each cell of the matrix multiply includes: a weight matrix register configured to receive a weight input from either a transposed or a non-transposed weight shift register; a transposed weight shift register configured to receive a weight input from a horizontal direction to be stored in the weight matrix register; a non-transposed weight shift register configured to receive a weight input from a vertical direction to be stored in the weight matrix register; and a multiply unit that is coupled to the weight matrix register and configured to multiply the weight input of the weight matrix register with a vector data input in order to obtain a multiplication result.

Claims20

Dependent claims

A processor comprising:

one or more cores, each core having an array of cells, and
one or more storage devices on which are stored instructions that are operable, when executed by a core of the one or more cores, to cause the core to perform operations comprising:
receiving data representing instructions for performing a portion of operations;
generating instructions that, for each cell of an array of cells included in the core and once executed by the cell, causes the cell to perform operations comprising:
receiving, by a weight matrix register of the cell, a weight input from a plurality of weight storing registers, wherein each of the plurality of weight storing registers is configured to receive a respective weight input from a respective direction of the array, wherein the respective directions comprise a first direction of the array and a second direction of the array different from the first direction;
and
obtaining, by a multiply unit that is coupled to the weight matrix register, a respective multiplication result based on the received weight input and another input;
and
generating a result for performing the portion of operations by combining the respective multiplication results obtained from the array of cells.

6 more elements

A method performed by a core of one or more cores included in a processor, the method comprising:

receiving data representing instructions for performing a portion of operations;
generating instructions that, for each cell of an array of cells included in the core and once executed by the cell, causes the cell to perform operations comprising:
receiving, by a weight matrix register of the cell, a weight input from a plurality of weight storing registers, wherein each of the plurality of weight storing registers is configured to receive a respective weight input from a respective direction of the array, wherein the respective directions comprise a first direction of the array and a second direction of the array different from the first direction;
and
obtaining, by a multiply unit that is coupled to the weight matrix register, a respective multiplication result based on the received weight input and another input;
and
generating a result for performing the portion of operations by combining the respective multiplication results obtained from the array of cells.

4 more elements

A non-transitory computer program product storing instructions that, when executed by a core of one or more cored included in a programmable processor, cause the core to perform operations comprising:

receiving data representing instructions for performing a portion of operations;
generating instructions that, for each cell of an array of cells included in the core and once executed by the cell, causes the cell to perform operations comprising:
receiving, by a weight matrix register of the cell, a weight input from a plurality of weight storing registers, wherein each of the plurality of weight storing registers is configured to receive a respective weight input from a respective direction of the array, wherein the respective directions comprise a first direction of the array and a second direction of the array different from the first direction;
and
obtaining, by a multiply unit that is coupled to the weight matrix register, a respective multiplication result based on the received weight input and another input;
and
generating a result for performing the portion of operations by combining the respective multiplication results obtained from the array of cells.

4 more elements

Dependencies are inferred from the claim text.

Facts

Application: 18/111,468Filed 17 Feb 2023 · continuation
Priority: 17 May 2017
Published: US 2023/0267172 A124 Aug 2023
Granted: 20 Feb 2024
Term ends: Not estimatedThe source records don’t support an estimate.

Classification12 codes

G06F 17/16
Complex mathematical operations: matrix or vector computation , e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
G06F 5/015
For shifting, e.g. justifying, scaling, normalising: having at least two separately controlled shifting levels, e.g. using shifting matrices
G06F 7/5443
For evaluating functions by calculation: sum of products
G06F 9/30036
To perform operations on data operands: instructions to perform operations on packed data, e.g. vector, tile or matrix operations

Family

Application62/507,766Us provisional application · filed 17 May 2017
Application15/983,037Parent of this continuation · filed 17 May 2018
Application16/529,662Parent of this continuation · filed 1 Aug 2019
Application16/915,286Parent of this continuation · filed 29 Jun 2020
US 10,698,976 B2Parent of this continuation · filed 30 Jun 2020
US 10,698,974 B2Parent of this continuation · filed 30 Jun 2020
Application17/210,293Parent of this continuation · filed 23 Mar 2021
US 10,970,362 B2Parent of this continuation · filed 6 Apr 2021
US 2023/0267172 A1Pre-grant publication · published 24 Aug 2023
US 11,599,601 B2Parent of this continuation
US 2018/0336163 A1Published application
US 2019/0354571 A1Published application
US 2020/0327186 A1Published application
US 2021/0209193 A1Published application
US 11,907,330 B2This patent

From the US filing history. Foreign family members aren’t listed.

Timeline

Citations

Open elsewhere

Building an agent or pulling many patents? Use the API or MCP server instead of scraping pages. Get an API key