# US 12,217,056 B2 · Load/store unit for a tensor engine and methods for loading or storing a tensor

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,217,056 B2
- Document ID: US12217056B2
- Status: Granted
- Kind: B2
- Application number: 18/423,210
- Filed: 2024-01-25
- Published: 2024-12-05
- Granted: 2025-02-04
- Earliest claimed: 2023-01-27
- Assignees: Celestial AI Inc.
- Applicants: Celestial AI Inc.
- Inventors: Philip Winterbottom
- Web page: https://patentgrep.com/patent/US12217056B2

## CPC

- G06F 9/30036: {Instructions to perform operations on packed data, e.g. vector, tile or matrix operations}
- G06F 17/16: Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)}
- G06F 9/30065: {Loop control instructions; iterative instructions, e.g. LOOP, REPEAT}
- G06F 9/3013: {according to data content, e.g. floating-point registers, address registers}
- G06N 3/063: using electronic means
- G06N 3/08: Learning methods
- G06N 20/10: using kernel methods, e.g. support vector machines [SVM]
- G06N 20/00: Machine learning
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A method for processing a tensor is described including obtaining a first register for a number of items in the tensor. One or more second registers for a number of items in a first and a second axis of the tensor are obtained. A stride in the first and the second axis is obtained A next item in the tensor is obtained using the stride in the first axis and a first offset register, when the first register indicates the tensor has additional items to process and the second registers indicate the next item resides in the first axis. A next item in the tensor is obtained using the stride in the first axis and the second axis, the first offset register, and a second offset register. The first register and a second register is modified. The first and the second offset registers are modified.

## Claims

### Claim 1

A method for processing a tensor, comprising:
obtaining a first register for a number of items in the tensor;
obtaining one or more second registers for a number of items in a first and a second axis of the tensor;
obtaining a stride in the first and the second axis;
obtaining a next item in the tensor using the stride in the first axis and a first offset register, when the first register indicates the tensor has additional items to process and the second registers indicate the next item resides in the first axis;
obtaining a next item in the tensor using the stride in the first axis and the second axis, the first offset register, and a second offset register, when the first register indicates the tensor has additional items to process, and the second registers indicate the next item resides in the second axis of the tensor;
modifying the first register and one or more of the second registers; and
modifying at least one of the first and the second offset registers.

### Claim 2 (depends on 1)

The method of claim 1, further comprising determining an effective address of the next item using a base address for the tensor in a memory region and one or more values stored in the first and the second offset registers.

### Claim 3 (depends on 2)

The method of claim 2, further comprising reading the next item at the effective address.

### Claim 4 (depends on 2)

The method of claim 2, further comprising writing the next item to the effective address.

### Claim 5 (depends on 1)

The method of claim 1, wherein obtaining a next item in the tensor in a first axis further comprises determining an order for traversing the tensor.

### Claim 6 (depends on 2)

The method of claim 2, wherein determining an effective address of the next item using a base address for the tensor in a memory region and one or more values stored in the first and the second offset registers further comprises:
setting a channel, a height, or a width as the first axis.

### Claim 7 (depends on 1)

The method of claim 1, wherein obtaining a stride uses a primitive data type associated with the tensor.

### Claim 8 (depends on 7)

The method of claim 7, wherein the primitive data type is selected from the group consisting of bits, bytes, integers, words, boolean values, (brain floating-point) BF-16, or FP-32.

### Claim 9 (depends on 1)

The method of claim 1, wherein modifying the first register and one or more of the second registers includes incrementing or decrementing the first register and one or more of the second registers.

### Claim 10 (depends on 1)

The method of claim 1, wherein modifying at least one of the first and the second offset registers includes one or more of adding a first value in the first offset register to the stride in the first axis and adding a second value in the second offset register to the stride in the second axis.

### Claim 11

A load/store unit (LDSU) for a tensor engine comprising:
a first register for storing values associated with a number of items in a tensor;
a plurality of second registers for storing a number of items in a first and a second axis of the tensor;
a plurality of offset registers associated with the first and the second axis;
a first and a second stride register associated with the first and the second axis;
a tensor walking module configured to obtain a next item in the tensor using a first stride register and a first offset register, when the first register indicates the tensor has additional items to process and the second registers indicate the next item resides in the first axis of the tensor,
the tensor walking module further configured to obtain a next item in the tensor using the first and the second stride registers, the first offset register, and a second offset register, when the first register indicates the tensor has additional items to process, and the second registers indicate the next item resides in the second axis of the tensor;
an iteration tracking module configured to modify the first and the second registers; and
a striding module configured to modify at least one of the first offset register or the second offset register.

### Claim 12 (depends on 11)

The LDSU of claim 11, comprising an address module configured to determine an effective address of the next item using a base address for the tensor in a memory region and one or more values stored in the first and the second offset registers.

### Claim 13 (depends on 12)

The LDSU of claim 12, further comprising an access module is configured to read the next item at the effective address.

### Claim 14 (depends on 13)

The LDSU of claim 13, wherein the access module is further configured to write the next item to the effective address.

### Claim 15 (depends on 11)

The LDSU of claim 11, further comprising a layout module is used to determine an order for traversing a plurality of dimensions of the tensor.

### Claim 16 (depends on 15)

The LDSU of claim 15, wherein a first dimension of the plurality of dimensions is selected from the group consisting of a channel, a height, and a width.

### Claim 17 (depends on 11)

The LDSU of claim 11, wherein the striding module uses an item type associated with the tensor to modify the first offset register or the second offset register.

### Claim 18 (depends on 17)

The LDSU of claim 17, wherein the item type is selected from the group consisting of bits, bytes, integers, words, boolean values, (brain floating-point) BF-16, or FP-32.

### Claim 19 (depends on 11)

The LDSU of claim 11, wherein the iteration tracking module modifies the first and second registers by incrementing or decrementing values in the first and the second registers.

### Claim 20 (depends on 11)

The LDSU of claim 11, wherein the striding module modifies at least one of the first offset register or the second offset register by adding a first value in the first offset register to a second value in the first stride register and adding a third value in the second offset register to a fourth value in the second stride register.