# US 2019/0171450 A1 · Programmable matrix processing engine

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0171450 A1
- Document ID: US20190171450A1
- Status: Published application
- Kind: A1
- Application number: 16/264,483
- Filed: 2019-01-31
- Published: 2019-06-06
- Earliest claimed: 2016-12-30
- Assignees: Intel Corporation
- Applicants: Intel Corporation
- Inventors: Tony L. Werner; Aravind Kalaiah; Vijay Korthikanti; Horace Lau
- Web page: https://patentgrep.com/patent/US20190171450A1

## CPC

- G06F 9/3001: {Arithmetic instructions}
- G06F 9/30043: {LOAD or STORE instructions; Clear instruction}
- G06N 3/063: using electronic means
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

In one embodiment, a matrix operation may be performed on one or more matrix operands. For example, matrix data may be received from a multi-dimensional memory, wherein the matrix data is associated with the one or more matrix operands. The one or more matrix operands may be extracted from the matrix data. A matrix routine associated with the matrix operation may be identified. The matrix routine may be executed on a matrix processor using the one or more matrix operands. A result of the matrix operation may be obtained based on the matrix routine executed by the matrix processor.

## Claims

### Claim at position 1

1-25. (canceled)

### Claim 26

A matrix processing circuit, comprising:
a matrix routine memory comprising circuitry to store a plurality of programmable matrix routines, wherein each of the plurality of programmable matrix routines comprises instructions for performing a corresponding matrix operation of a plurality of matrix operations;
one or more matrix processing units comprising circuitry to perform matrix computations; and
a controller comprising circuitry to:
receive a command to perform a particular matrix operation of the plurality of matrix operations;
identify a programmable matrix routine corresponding to the particular matrix operation, wherein the programmable matrix routine is identified from the plurality of programmable matrix routines;
receive the programmable matrix routine from the matrix routine memory;
execute the programmable matrix routine, wherein one or more matrix computations associated with execution of the programmable matrix routine are performed using the one or more matrix processing units; and
determine a result of the particular matrix operation, wherein the result is determined based on execution of the programmable matrix routine.

### Claim 27 (depends on 26)

The matrix processing circuit of claim 26, wherein the circuitry to execute the programmable matrix routine is to:
receive one or more matrix operands associated with the particular matrix operation; and
perform, using the one or more matrix processing units, the one or more matrix computations on the one or more matrix operands.

### Claim 28 (depends on 27)

The matrix processing circuit of claim 27, wherein:
the matrix processing circuit further comprises a plurality of memory resource blocks; and
the circuitry to receive the one or more matrix operands associated with the particular matrix operation is further to:
receive matrix data from a memory, wherein the matrix data is associated with the one or more matrix operands;
extract the one or more matrix operands from the matrix data; and
store the one or more matrix operands in one or more of the plurality of memory resource blocks.

### Claim 29 (depends on 28)

The matrix processing circuit of claim 28, wherein:
the programmable matrix routine comprises a set of instructions for performing the particular matrix operation; and
the set of instructions comprises:
a first subset of instructions to receive the matrix data from the memory and extract the one or more matrix operands from the matrix data; and
a second subset of instructions to perform the one or more matrix computations on the one or more matrix operands.

### Claim 30 (depends on 28)

The matrix processing circuit of claim 28, wherein:
the one or more matrix processing units comprise a plurality of matrix processing units; and
the matrix processing circuit further comprises a matrix processing cluster, wherein the matrix processing cluster comprises the plurality of matrix processing units and the plurality of memory resource blocks.

### Claim 31 (depends on 26)

The matrix processing circuit of claim 26, wherein the controller further comprises circuitry to:
receive the plurality of programmable matrix routines from a host computing system; and
store the plurality of programmable matrix routines in the matrix routine memory.

### Claim 32 (depends on 26)

The matrix processing circuit of claim 26, wherein the one or more matrix computations comprise one or more matrix multiplication computations.

### Claim 33 (depends on 26)

The matrix processing circuit of claim 26, wherein the one or more matrix computations comprise one or more convolution computations.

### Claim 34 (depends on 26)

The matrix processing circuit of claim 26, wherein the particular matrix operation is associated with an operation for an artificial neural network.

### Claim 35

A system, comprising:
a processor to execute an application, wherein execution of the application comprises a plurality of matrix operations; and
matrix processing circuitry to perform the plurality of matrix operations, wherein the matrix processing circuitry comprises:
a matrix routine memory comprising circuitry to store a plurality of programmable matrix routines, wherein each of the plurality of programmable matrix routines comprises instructions for performing a corresponding matrix operation of the plurality of matrix operations;
one or more matrix processing clusters, wherein each of the one or more matrix processing clusters comprises a plurality of matrix processing units, wherein the plurality of matrix processing units comprise circuitry to perform matrix computations; and
a controller comprising circuitry to:
receive a command to perform a particular matrix operation of the plurality of matrix operations;
identify a programmable matrix routine corresponding to the particular matrix operation, wherein the programmable matrix routine is identified from the plurality of programmable matrix routines;
receive the programmable matrix routine from the matrix routine memory;
distribute execution of the programmable matrix routine across the one or more matrix processing clusters; and
determine a result of the particular matrix operation, wherein the result is determined based on execution of the programmable matrix routine.

### Claim 36 (depends on 35)

The system of claim 35, wherein the one or more matrix processing clusters comprise:
a plurality of matrix processing clusters; and
a multi-dimensional mesh interconnect to communicatively couple the plurality of matrix processing clusters.

### Claim 37 (depends on 35)

The system of claim 35, wherein each of the one or more matrix processing clusters further comprises circuitry to:
receive one or more matrix operands associated with the particular matrix operation; and
perform, using the plurality of matrix processing units, a plurality of matrix computations on the one or more matrix operands.

### Claim 38 (depends on 37)

The system of claim 37, wherein:
the matrix processing circuitry further comprises one or more memory modules;
each of the one or more matrix processing clusters further comprises a plurality of memory resource blocks; and
the circuitry to receive the one or more matrix operands associated with the particular matrix operation is further to:
receive matrix data from the one or more memory modules, wherein the matrix data is associated with the one or more matrix operands;
extract the one or more matrix operands from the matrix data; and
store the one or more matrix operands in one or more of the plurality of memory resource blocks.

### Claim 39 (depends on 38)

The system of claim 38, wherein:
the programmable matrix routine comprises a set of instructions for performing the particular matrix operation; and
the set of instructions comprises:
a first subset of instructions to receive the matrix data from the one or more memory modules and extract the one or more matrix operands from the matrix data; and
a second subset of instructions to perform the plurality of matrix computations on the one or more matrix operands.

### Claim 40 (depends on 35)

The system of claim 35, wherein the controller further comprises circuitry to:
receive the plurality of programmable matrix routines from the processor; and
store the plurality of programmable matrix routines in the matrix routine memory.

### Claim 41

At least one non-transitory machine accessible storage medium having instructions stored thereon, the instructions, when executed on a machine, cause the machine to:
receive a command to perform a particular matrix operation of a plurality of matrix operations;
identify a programmable matrix routine corresponding to the particular matrix operation, wherein the programmable matrix routine is identified from a plurality of programmable matrix routines, wherein each of the plurality of programmable matrix routines comprises programmable instructions for performing a corresponding matrix operation of the plurality of matrix operations;
receive the programmable matrix routine from a matrix routine memory, wherein the plurality of programmable matrix routines are stored on the matrix routine memory;
execute the programmable matrix routine, wherein one or more matrix computations associated with execution of the programmable matrix routine are performed using one or more matrix processing units, wherein the one or more matrix processing units comprise circuitry to perform matrix computations; and
determine a result of the particular matrix operation, wherein the result is determined based on execution of the programmable matrix routine.

### Claim 42 (depends on 41)

The storage medium of claim 41, wherein the instructions that cause the machine to execute the programmable matrix routine further cause the machine to:
receive one or more matrix operands associated with the particular matrix operation; and
perform, using the one or more matrix processing units, the one or more matrix computations on the one or more matrix operands.

### Claim 43 (depends on 42)

The storage medium of claim 42, wherein the instructions that cause the machine to receive the one or more matrix operands associated with the particular matrix operation further cause the machine to:
receive matrix data from a memory, wherein the matrix data is associated with the one or more matrix operands;
extract the one or more matrix operands from the matrix data; and
store the one or more matrix operands in one or more of a plurality of memory resource blocks.

### Claim 44 (depends on 43)

The storage medium of claim 43, wherein:
the programmable matrix routine comprises a set of instructions for performing the particular matrix operation; and
the set of instructions comprises:
a first subset of instructions to receive the matrix data from the memory and extract the one or more matrix operands from the matrix data; and
a second subset of instructions to perform the one or more matrix computations on the one or more matrix operands.

### Claim 45 (depends on 41)

The storage medium of claim 41, wherein the instructions further cause the machine to:
receive the plurality of programmable matrix routines from a host computing system; and
store the plurality of programmable matrix routines in the matrix routine memory.

### Claim 46

A method, comprising:
receiving a command to perform a particular matrix operation of a plurality of matrix operations;
identifying a programmable matrix routine corresponding to the particular matrix operation, wherein the programmable matrix routine is identified from a plurality of programmable matrix routines, wherein each of the plurality of programmable matrix routines comprises programmable instructions for performing a corresponding matrix operation of the plurality of matrix operations;
receiving the programmable matrix routine from a matrix routine memory, wherein the plurality of programmable matrix routines are stored on the matrix routine memory;
executing the programmable matrix routine, wherein one or more matrix computations associated with execution of the programmable matrix routine are performed using one or more matrix processing units, wherein the one or more matrix processing units comprise circuitry to perform matrix computations; and
determining a result of the particular matrix operation, wherein the result is determined based on execution of the programmable matrix routine.

### Claim 47 (depends on 46)

The method of claim 46, wherein executing the programmable matrix routine comprises:
receiving one or more matrix operands associated with the particular matrix operation; and
performing, using the one or more matrix processing units, the one or more matrix computations on the one or more matrix operands.

### Claim 48 (depends on 47)

The method of claim 47, wherein receiving the one or more matrix operands associated with the particular matrix operation comprises:
receiving matrix data from a memory, wherein the matrix data is associated with the one or more matrix operands;
extracting the one or more matrix operands from the matrix data; and
storing the one or more matrix operands in one or more of a plurality of memory resource blocks.

### Claim 49 (depends on 48)

The method of claim 48, wherein:
the programmable matrix routine comprises a set of instructions for performing the particular matrix operation; and
the set of instructions comprises:
a first subset of instructions to receive the matrix data from the memory and extract the one or more matrix operands from the matrix data; and
a second subset of instructions to perform the one or more matrix computations on the one or more matrix operands.

### Claim 50 (depends on 46)

The method of claim 46, further comprising:
receiving the plurality of programmable matrix routines from a host computing system; and
storing the plurality of programmable matrix routines in the matrix routine memory.
