# US 2021/0334337 A1 · Implementing Fundamental Computational Primitives Using A Matrix Multiplication Accelerator (MMA)

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0334337 A1
- Document ID: US20210334337A1
- Status: Published application
- Kind: A1
- Application number: 17/367,389
- Filed: 2021-07-04
- Published: 2021-10-28
- Earliest claimed: 2017-03-01
- Assignees: Texas Instruments Incorporated
- Applicants: Texas Instruments Incorporated
- Inventors: Arthur John Redfern; Timothy David Anderson; Kai Chirca; Chenchi Luo; Zhenhua Yu
- Web page: https://patentgrep.com/patent/US20210334337A1

## CPC

- G06F 17/16: Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)}
- G06F 17/141: {Discrete Fourier transforms}
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/063: using electronic means
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A method for performing a fundamental computational primitive in a device is provided, where the device includes a processor and a matrix multiplication accelerator (MMA). The method includes configuring a streaming engine in the device to stream data for the fundamental computational primitive from memory, configuring the MMA to format the data, and executing the fundamental computational primitive by the device.

## Claims

### Claim 1

A matrix multiplication accelerator comprising:
a first formatting component having a first input and a first output;
a second formatting component having a second input and second output;
a third formatting component having a third input and a third output;
a row offset component having an offset input coupled to the second output, and an offset output;
a first memory having a first matrix input coupled to the first output, and a first matrix output;
a second memory having a second matrix input coupled to the offset output, and a second matrix output;
a nonlinearity component having a nonlinearity input, and a nonlinearity output coupled to the third input;
a third memory having a third matrix input, and a third matrix output coupled to the nonlinearity input; and
a matrix multiplication component coupled to the first matrix output, the second matrix output, and the third matrix input.

### Claim 2 (depends on 1)

The matrix multiplication accelerator of claim 1, wherein:
the first input, the second input, and the third output are coupled to a processor having a first streaming engine and a second streaming engine.

### Claim 3 (depends on 1)

The matrix multiplication accelerator of claim 1, wherein:
the first formatting component, the second formatting component, the third formatting component, the row offset component, and the nonlinearity component are each include a configuration input.

### Claim 4 (depends on 3)

The matrix multiplication accelerator of claim 3, wherein:
the first formatting component, the second formatting component, the third formatting component, the row offset component, and the nonlinearity component are each configured based on a configuration setting received on the configuration input.

### Claim 5 (depends on 4)

The matrix multiplication accelerator of claim 4, wherein:
the configuration setting is based on a computational primitive.

### Claim 6 (depends on 5)

The matrix multiplication accelerator of claim 5, wherein:
the computational primitive includes one of a two-dimensional (2D) convolution, a matrix multiplication, a matrix point wise multiplication, a matrix addition, a vector matrix multiplication, a vector summation, an affine transformation, a fast Fourier transform, a discrete cosign transform, a correlation, a matrix assignment, a matrix permutation, and a matrix transposition.

### Claim 7 (depends on 1)

The matrix multiplication accelerator of claim 1, wherein:
the second memory includes two 32×32 multiplicand buffers.

### Claim 8 (depends on 1)

The matrix multiplication accelerator of claim 1, wherein:
the third memory includes two 32×32 result buffers.

### Claim 9 (depends on 1)

The matrix multiplication accelerator of claim 1, wherein:
the matrix multiplication accelerator performs a load operation, a store operation, and an execute operation in a single cycle.

### Claim 10 (depends on 9)

The matrix multiplication accelerator of claim 9, wherein:
the load operation includes a first field identifying a location of a first data to load into the first memory.

### Claim 11 (depends on 9)

The matrix multiplication accelerator of claim 9, wherein:
the load operation includes a second field identifying a location of a second data to load into the second memory.

### Claim 12 (depends on 9)

The matrix multiplication accelerator of claim 9, wherein:
the load operation includes a third field identifying whether the load operation is to be performed.

### Claim 13 (depends on 9)

The matrix multiplication accelerator of claim 9, wherein:
the store operation includes a first field identifying a location in the third memory that stores a matrix multiplication output received on the third matrix input from the matrix multiplication component.

### Claim 14 (depends on 9)

The matrix multiplication accelerator of claim 9, wherein:
the store operation includes a second field identifying whether the store operation is to be performed.

### Claim 15 (depends on 9)

The matrix multiplication accelerator of claim 9, wherein:
the execute operation includes a first field identifying an operation type.

### Claim 16 (depends on 1)

The matrix multiplication accelerator of claim 1, wherein:
the nonlinearity output is zero when the nonlinearity input is negative.

### Claim 17 (depends on 1)

The matrix multiplication accelerator of claim 1, wherein:
the nonlinearity output is set to the nonlinearity input when the nonlinearity input is non-negative.

### Claim 18

A method comprising:
configuring, by a processor, a matrix multiplication accelerator (MMA) based on a computational primitive; and
executing, by the MMA, the computational primitive to produce an MMA output.

### Claim 19 (depends on 18)

The method of claim 18, wherein:
wherein executing the computational primitive further comprises loading data into a first matrix buffer concurrently with execution of matrix multiplication using data in a second matrix buffer.

### Claim 20 (depends on 19)

The method of claim 19, wherein:
the second matrix buffer includes a foreground buffer and a background buffer.
