# US 12,007,904 B2 · Method and apparatus for vector based matrix multiplication

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,007,904 B2
- Document ID: US12007904B2
- Status: Granted
- Kind: B2
- Application number: 17/749,671
- Filed: 2022-05-20
- Published: 2022-09-08
- Granted: 2024-06-11
- Earliest claimed: 2019-05-24
- Assignees: Texas Instruments Incorporated
- Applicants: Texas Instruments Incorporated
- Inventors: Asheesh Bhardwaj; Mujibur Rahman; Timothy David Anderson
- Web page: https://patentgrep.com/patent/US12007904B2

## CPC

- G06F 12/1045: associated with a data cache
- G06F 7/24: Sorting, i.e. extracting data from one or more carriers, rearranging the data in numerical or other ordered sequence, and rerecording the sorted data on the original carrier or on a different carrier or set of carriers {sorting methods in general}(G06F7/36 takes precedence)
- G06F 7/483: Computations with numbers represented by a non-linear combination of denominational numbers, e.g. rational numbers, logarithmic number system or floating-point numbers {(G06F7/4806, G06F7/4824, G06F7/49, G06F7/491, G06F7/544 take precedence)}
- G06F 7/487: Multiplying; Dividing {(G06F7/4833, G06F7/4836 take precedence)}
- G06F 7/4876: {Multiplying}
- G06F 7/49915: {Mantissa overflow or underflow in handling floating-point numbers}
- G06F 7/53: in parallel-parallel fashion, i.e. both operands being entered in parallel (G06F7/533 takes precedence)
- G06F 7/57: Arithmetic logic units [ALU], i.e. arrangements or devices for performing two or more of the operations covered by groups G06F7/483 – G06F7/556 or for performing logical operations {(G06F7/49, G06F7/491 take precedence)}
- G06F 9/3001: {Arithmetic instructions}
- G06F 9/30014: {with variable precision}
- G06F 9/30018: {Bit or string instructions}
- G06F 9/30021: {Compare instructions, e.g. Greater-Than, Equal-To, MINMAX}
- G06F 9/30032: {Movement instructions, e.g. MOVE, SHIFT, ROTATE, SHUFFLE}
- G06F 9/30036: {Instructions to perform operations on packed data, e.g. vector, tile or matrix operations}
- G06F 9/3004: {to perform operations on memory}
- G06F 9/30065: {Loop control instructions; iterative instructions, e.g. LOOP, REPEAT}
- G06F 9/30072: {to perform conditional operations, e.g. using predicates or guards}
- G06F 9/3009: {Thread control instructions}
- G06F 9/30098: {Register arrangements}
- G06F 9/30112: {comprising data of variable length}
- G06F 9/30145: {Instruction analysis, e.g. decoding, instruction word fields}
- G06F 9/30149: {of variable length instructions}
- G06F 9/3016: {Decoding the operand specifier, e.g. specifier format}
- G06F 9/32: Address formation of the next instruction, e.g. by incrementing the instruction counter (G06F9/38 takes precedence)
- G06F 9/345: of multiple operands or results {(addressing multiple banks G06F12/06)}
- G06F 9/3802: {Instruction prefetching}
- G06F 9/3818: {Decoding for concurrent execution}
- G06F 9/383: {Operand prefetching (cache prefetching G06F12/0862)}
- G06F 9/3836: {Instruction issuing, e.g. dynamic instruction scheduling or out of order instruction execution}
- G06F 9/3851: {from multiple instruction streams, e.g. multistreaming}
- G06F 9/3856: {Reordering of instructions, e.g. using queues or age tags}
- G06F 9/3867: {using instruction pipelines}
- G06F 9/3887: {controlled by a single instruction for multiple data lanes [SIMD]}
- G06F 9/48: Program initiating; Program switching, e.g. by interrupt
- G06F 11/00: Error detection; Error correction; Monitoring (error detection, correction or monitoring in information storage based on relative movement between record carrier and transducer G11B20/18; monitoring, i.e. supervising the progress of recording or reproducing G11B27/36; in static stores G11C29/00)
- G06F 11/1048: {using arrangements adapted for a specific error detection or correction feature}
- G06F 12/0862: with prefetch
- G06F 12/0875: with dedicated cache, e.g. instruction or stack
- G06F 12/0897: with two or more cache hierarchy levels (with multilevel cache hierarchies G06F12/0811)
- G06F 12/1009: using page tables, e.g. page table structures
- G06F 17/16: Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)}
- H03H 17/06: Non-recursive filters
- H03H 17/0664: {where the output-delivery frequency is lower than the input sampling frequency, i.e. decimation}
- G06F 9/325: {for loops, e.g. loop detection or loop counter}
- G06F 9/381: {Loop buffering}
- G06F 9/3822: {Parallel decoding, e.g. parallel decode units}
- G06F 11/10: Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
- G06F 15/7807: {System on chip, i.e. computer system on a single chip; System in package, i.e. computer system on one or more chips in a single package}
- G06F 15/781: {On-chip cache; Off-chip memory}
- G06F 2212/452: Instruction code
- G06F 2212/60: Details of cache memory
- G06F 2212/602: Details relating to cache prefetching
- G06F 2212/68: Details of translation look-aside buffer [TLB]
- H03H 2017/0298: {DSP implementation}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H03H: IMPEDANCE NETWORKS, e.g. RESONANT CIRCUITS; RESONATORS (waveguides, resonators, lines or other devices of the waveguide type H01P)

## Abstract

A method is provided that includes performing, by a processor in response to a vector matrix multiply instruction, multiplying an m×n matrix (A matrix) and a n×p matrix (B matrix) to generate elements of an m×p matrix (R matrix), and storing the elements of the R matrix in a storage location specified by the vector matrix multiply instruction.

## Claims

### Claim 1

A system comprising:
a first buffer configured to receive a first set of data elements;
a first permute component coupled to the first buffer;
a second buffer configured to receive a second set of data elements;
a second permute component coupled to the second buffer;
a multiplexer configured to select between the first permute component and the second permute component based on which of the first buffer and the second buffer stores a next data element of the first set of data elements or the second set of data elements, wherein the multiplexer includes:
a first input coupled to the first permute component;
a second input coupled to the second permute component; and
an output; and
a multiplication component coupled to the output of the multiplexer.

### Claim 2 (depends on 1)

The system of claim 1, wherein:
the first permute component includes a first permute network configured to permute a set of data elements for a first instruction; and
the second permute component includes a second permute network configured to permute the set of data elements for a second instruction.

### Claim 3 (depends on 2)

The system of claim 2, wherein:
the output of the multiplexer is a first output;
the first permute network includes a second output coupled to the first input of the multiplexer;
the second permute network includes a third output coupled to the second input of the multiplexer; and
the multiplexer is configured to select the second output or the third output based on a selection signal from an instruction decoder.

### Claim 4 (depends on 1)

The system of claim 1, further comprising a logical OR circuit coupled to the output of the multiplexer and the multiplication component.

### Claim 5 (depends on 4)

The system of claim 4, wherein:
the multiplexer is a first multiplexer;
the logical OR circuit is coupled to a second multiplexer; and
the logical OR circuit is configured to select the first multiplexer or the second multiplexer.

### Claim 6 (depends on 5)

The system of claim 5, wherein:
the logical OR circuit is configured to concatenate a first output of the first multiplexer and a second output of the second multiplexer.

### Claim 7 (depends on 1)

The system of claim 1, wherein:
the multiplication component is configured to perform vector multiplication.

### Claim 8 (depends on 1)

The system of claim 1,
wherein the first permute component is configured to permute a set of data elements to generate permutated data, and
wherein the multiplication component is configured to multiply the permutated data.

### Claim 9 (depends on 1)

The system of claim 1, wherein input packets of the first set of data elements and the second set of data elements are alternated in a ping pong fashion between the first buffer and the second buffer.

### Claim 10 (depends on 1)

The system of claim 1, further comprising an instruction decoder circuit configured to:
generate a select signal based on which buffer of the first buffer and the second buffer stores the next data element; and
send the select signal to the multiplexer.

### Claim 11

A method comprising:
receiving, at a first buffer of a data load interface, a first set of data elements, wherein the data load interface includes a first permute component coupled to the first buffer;
permutating, by the first permute component, the first set of data elements to generate first permutated data;
receiving, at a second buffer of the data load interface, a second set of data elements, wherein the data load interface further includes a second permute component coupled to the second buffer;
permutating, by the second permute component, the second set of data elements to generate second permutated data;
selecting, by a multiplexer coupled to the first permute component and the second permute component, between the first permutated data and the second permutated data based on which of the first buffer and the second buffer stores a next data element of the first set of data elements or the second set of data elements; and
multiplying, by a multiplication component coupled to the multiplexer, the first permutated data or the second permutated data.

### Claim 12 (depends on 11)

The method of claim 11, wherein:
the first permute component includes a first permute network configured to permute a set of data elements for a first instruction; and
the second permute component includes a second permute network configured to permute the set of data elements for a second instruction.

### Claim 13 (depends on 12)

The method of claim 12, wherein:
the multiplexer is coupled to a first output of the first permute network and a second output of the second permute network; and
the multiplexer is configured to select the first output or the second output based on a selection signal from an instruction decoder.

### Claim 14 (depends on 11)

The method of claim 11, wherein:
the data load interface is a first data load interface; and
a logical OR circuit couples the first data load interface and a second data load interface to the multiplication component.

### Claim 15 (depends on 14)

The method of claim 14, wherein:
selecting, by the logical OR circuit, the first data load interface or the second data load interface.

### Claim 16 (depends on 14)

The method of claim 14, wherein:
concatenating, by the logical OR circuit, a first output of the first data load interface and a second output of the second data load interface.

### Claim 17 (depends on 11)

The method of claim 11, wherein:
the multiplication component is configured to perform vector multiplication.

### Claim 18

A system comprising:
a first buffer;
a first permute component coupled to the first buffer;
a second buffer;
a second permute component coupled to the second buffer;
a third buffer;
a third permute component coupled to the third buffer;
a fourth buffer;
a fourth permute component coupled to the fourth buffer;
a first multiplexer coupled to the first and second permute components;
a second multiplexer coupled to the third and fourth permute components;
a logical OR circuit coupled to the first multiplexer and the second multiplexer;
a multiplication component coupled to the logical OR circuit.

### Claim 19 (depends on 18)

The system of claim 18,
wherein the first multiplexer is configured to select between the first permute component and the second permute component based on which of the first buffer and the second buffer stores a next data packet for the multiplication component, and
wherein the second multiplexer is configured to select between the third permute component and the fourth permute component based on which of the third buffer and the fourth buffer stores a next data packet for the multiplication component.

### Claim 20 (depends on 18)

The system of claim 18,
wherein the logical OR circuit is configured to select the first multiplexer or the second multiplexer, and
wherein the logical OR circuit is configured to concatenate a first output of the first multiplexer and a second output of the second multiplexer.