# US 12,118,358 B2 · One-dimensional zero padding in a stream of matrix elements

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,118,358 B2
- Document ID: US12118358B2
- Status: Granted
- Kind: B2
- Application number: 17/583,380
- Filed: 2022-01-25
- Published: 2022-05-12
- Granted: 2024-10-15
- Earliest claimed: 2013-07-15
- Assignees: Texas Instruments Incorporated
- Applicants: Texas Instruments Incorporated
- Inventors: Son Hung Tran; Shyam Jagannathan; Timothy David Anderson
- Web page: https://patentgrep.com/patent/US12118358B2

## CPC

- G06F 9/3016: {Decoding the operand specifier, e.g. specifier format}
- G06F 9/30014: {with variable precision}
- G06F 9/30036: {Instructions to perform operations on packed data, e.g. vector, tile or matrix operations}
- G06F 9/30038: {using a mask}
- G06F 9/30043: {LOAD or STORE instructions; Clear instruction}
- G06F 9/30047: {Prefetch instructions; cache control instructions}
- G06F 9/30098: {Register arrangements}
- G06F 9/30101: {Special purpose registers}
- G06F 9/30112: {comprising data of variable length}
- G06F 9/30145: {Instruction analysis, e.g. decoding, instruction word fields}
- G06F 9/32: Address formation of the next instruction, e.g. by incrementing the instruction counter (G06F9/38 takes precedence)
- G06F 9/345: of multiple operands or results {(addressing multiple banks G06F12/06)}
- G06F 9/3455: {using stride}
- G06F 9/3802: {Instruction prefetching}
- G06F 9/3824: {Operand accessing}
- G06F 9/383: {Operand prefetching (cache prefetching G06F12/0862)}
- G06F 9/3867: {using instruction pipelines}
- G06F 11/00: Error detection; Error correction; Monitoring (error detection, correction or monitoring in information storage based on relative movement between record carrier and transducer G11B20/18; monitoring, i.e. supervising the progress of recording or reproducing G11B27/36; in static stores G11C29/00)
- G06F 11/1048: {using arrangements adapted for a specific error detection or correction feature}
- G06F 12/0811: with multilevel cache hierarchies
- G06F 12/0862: with prefetch
- G06F 12/0875: with dedicated cache, e.g. instruction or stack
- G06F 12/0897: with two or more cache hierarchy levels (with multilevel cache hierarchies G06F12/0811)
- G06F 15/8015: {One dimensional arrays, e.g. rings, linear arrays, buses}
- G06F 9/3822: {Parallel decoding, e.g. parallel decode units}
- G06F 11/10: Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
- G06F 17/16: Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)}
- G06F 2212/452: Instruction code
- G06F 2212/60: Details of cache memory
- G06F 2212/604: Details relating to cache allocation
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Software instructions are executed on a processor within a computer system to configure a streaming engine with stream parameters to define a multidimensional array. The stream parameters define a size for each dimension of the multidimensional array and a specified width for a selected dimension of the array. Data is fetched from a memory coupled to the streaming engine responsive to the stream parameters. A stream of vectors is formed for the multidimensional array responsive to the stream parameters from the data fetched from memory. When the selected dimension in the stream of vectors exceeds the specified width, the streaming engine inserts null elements into each portion of a respective vector for the selected dimension that exceeds the specified width in the stream of vectors. Stream vectors that are completely null are formed by the streaming engine without accessing the system memory for respective data.

## Claims

### Claim 1

A method comprising:
receiving a set of parameters that defines a size of an array in a dimension and defines a set of loops, wherein the set of loops includes a first loop;
for a first iteration of the first loop:
retrieving a first set of data from a memory;
for a second iteration of the first loop:
retrieving a second set of data from the memory; and
based on the size of the array in the dimension, replacing a portion of the second set of data with a first set of null elements to generate a third set of data;
for a third iteration of the first loop:
based on the size of the array in the dimension, generating a fourth set of data that includes a second set of null elements, wherein:
the generating of the second set of null elements of the fourth set of data does not include an access of the memory; and
providing the first set of data, the third set of data, and the fourth set of data.

### Claim 2 (depends on 1)

The method of claim 1, wherein the set of parameters specifies a null value for the first set of null elements and the second set of null elements.

### Claim 3 (depends on 1)

The method of claim 1, wherein each element of the first set of null elements and the second set of null elements has a value of zero.

### Claim 4 (depends on 1)

The method of claim 1, wherein:
the retrieving of the first set of data includes:
generating a first set of addresses;
translating the first set of addresses using a table look-aside buffer to generate a second set of addresses; and
retrieving the first set of data from the memory using the second set of addresses; and
the generating of the second set of null elements of the fourth set of data does not include an access of the table look-aside buffer.

### Claim 5 (depends on 1)

The method of claim 1, wherein the providing of the first set of data, the third set of data, and the fourth set of data includes providing a stream of vectors corresponding to the first set of data, the third set of data, and the fourth set of data.

### Claim 6 (depends on 5)

The method of claim 5, wherein the set of parameters specifies an element size of the array, a number of elements in each vector of the stream of vectors, and a number of vectors in the stream of vectors.

### Claim 7 (depends on 5)

The method of claim 5, wherein the receiving of the set of parameters, the retrieving of the first set of data, the retrieving of the second set of data, the generating of the fourth set of data, and the providing of the first set of data, the third set of data, and the fourth set of data are performed in response to a stream open command that specifies a register storing the set of parameters from among a set of registers.

### Claim 8 (depends on 1)

The method of claim 1, wherein the generating of the second set of null elements includes:
maintaining a count based on the size of the array in the dimension; and
generating a null element of the second set of null elements based on the count reaching zero.

### Claim 9 (depends on 1)

The method of claim 1, wherein:
the memory is an L2 cache memory; and
the retrieving of the first set of data retrieves the first set of data from the L2 cache memory by bypassing an L1 cache memory coupled to the L2 cache memory.

### Claim 10

A device comprising:
a register configured to store a set of parameters that defines a size of an array in a dimension and defines a loop; and
a memory configured to store the array;
a circuit coupled to the register and the memory and configured to:
for a first iteration of the loop:
retrieve a first set of data from the memory;
for a second iteration of the loop:
retrieve a second set of data from the memory; and
based on the size of the array in the dimension, replace a portion of the second set of data with a first set of null elements to produce generate a third set of data;
for a third iteration of the loop:
based on the size of the array in the dimension, generate a fourth set of data that includes a second set of null elements, wherein:
generation of the second set of null elements of the fourth set of data does not include an access of the memory; and
provide the first set of data, the third set of data, and the fourth set of data.

### Claim 11 (depends on 10)

The device of claim 10, wherein the set of parameters specifies a null value for the first set of null elements and the second set of null elements.

### Claim 12 (depends on 10)

The device of claim 10, wherein each element of the first set of null elements and the second set of null elements has a value of zero.

### Claim 13 (depends on 10)

The device of claim 10, wherein:
the circuit includes a table look-aside buffer;
the circuit is configured to retrieve the first set of data by:
generating a first set of addresses;
translating the first set of addresses using the table look-aside buffer to generate a second set of addresses; and
retrieving the first set of data from the memory using the second set of addresses; and
generation of the second set of null elements of the fourth set of data does not include an access of the table look-aside buffer.

### Claim 14 (depends on 10)

The device of claim 10, wherein the circuit is configured to provide the first set of data, the third set of data, and the fourth set of data as a stream of vectors.

### Claim 15 (depends on 14)

The device of claim 14, wherein the set of parameters specifies an element size of the array, a number of elements in each vector of the stream of vectors, and a number of vectors in the stream of vectors.

### Claim 16 (depends on 10)

The device of claim 10, wherein the circuit is configured to generate the second set of null elements by:
maintaining a count based on the size of the array in the dimension; and
generating a null element of the second set of null elements based on the count reaching zero.

### Claim 17

A device comprising:
a processor core comprising a register configured to store a set of parameters that defines a size of an array in a dimension and defines a loop;
a level one (L1) cache memory coupled to the processor core;
a level two (L2) cache memory coupled to the L1 cache memory; and
a circuit coupled between the processor core and the L2 cache memory in parallel with the L1 cache memory, wherein the circuit is configured to:
receive the set of parameters from the processor core;
for a first iteration of the loop, retrieve a first set of data from the L2 cache memory;
for a second iteration of the loop, based on the size of the array in the dimension, generate a second set of data that includes a set of null elements, wherein:
generation of the set of null elements of the second set of data does not include an access of the L2 cache memory; and
provide the first set of data and the second set of data.

### Claim 18 (depends on 17)

The device of claim 17, wherein:
the set of null elements is a first set of null elements; and
the circuit is configured to, for a third iteration of the loop:
retrieve a third set of data from the L2 cache memory; and
based on the size of the array in the dimension, replace a portion of the third data set of data with a second set of null elements.