# US 2026/0017060 A1 · Configuring a tensor operation pipeline in a hardware accelerator

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0017060 A1
- Document ID: US20260017060A1
- Status: Published application
- Kind: A1
- Application number: 18/769,245
- Filed: 2024-07-10
- Published: 2026-01-15
- Earliest claimed: 2024-07-10
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Ashraf Ayman MICHAIL; Li ZHANG; Nitin Naresh GAREGRAT; Thomas Craig SAVELL
- Web page: https://patentgrep.com/patent/US20260017060A1

## CPC

- G06F 9/30014: {with variable precision}
- G06F 9/3867: {using instruction pipelines}
- G06F 9/30036: {Instructions to perform operations on packed data, e.g. vector, tile or matrix operations}
- G06F 15/80: comprising an array of processing units with common control, e.g. single instruction multiple data processors (G06F15/82 takes precedence {; for correlation function computation G06F17/15})
- G06N 3/045: Combinations of networks
- G06N 3/048: Activation functions
- G06N 3/063: using electronic means
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A computing method is provided for configuring a tensor operation pipeline. In one example implementation, the method includes receiving a tensor operation pipeline definition and tensor data from a processor, at a configurable pipeline processing element array of a hardware accelerator. The method further includes, in each of a plurality of processing elements of the array, processing the tensor data by implementing a configurable tensor operation pipeline including one or more of the fixed tensor operation logic units according to the tensor operation pipeline definition. The method further includes outputting a tensor operation pipeline result based on the processing of the tensor data by each tensor operation pipeline in each processing element.

## Claims

### Claim 1

A computing method, comprising:
receiving a tensor operation pipeline definition and tensor data from a processor, at a configurable pipeline processing element array of a hardware accelerator;
in each of a plurality of processing elements of the array, processing the tensor data by implementing a configurable tensor operation pipeline including one or more of the fixed tensor operation logic units according to the tensor operation pipeline definition; and
outputting a tensor operation pipeline result based on the processing of the tensor data by each tensor operation pipeline in each processing element.

### Claim 2 (depends on 1)

The computing method of claim 1, wherein the configurable plurality of fixed tensor operation logic units are selected from the group consisting of a split logic unit, add logic unit, subtract logic unit, select logic unit, concatenate logic unit, and lookup logic unit.

### Claim 3 (depends on 1)

The computing method of claim 1, wherein
the tensor operation pipeline definition defines a plurality of stages, each stage specifying a corresponding one of the configurable plurality of fixed tensor operation logic units.

### Claim 4 (depends on 3)

The computing method of claim 3, wherein
the stages are in a predetermined order defined by an on-chip hardware layout, and individual fixed tensor operation logic units can be turned on or off by command.

### Claim 5 (depends on 1)

The computing method of claim 1, wherein
at least one of the stages includes a look up table logic unit as the fixed tensor operation logic unit for that stage.

### Claim 6 (depends on 1)

The computing method of claim 1, wherein
values for the look up table unit are included in the tensor operation pipeline definition.

### Claim 7 (depends on 1)

The computing method of claim 1, wherein
the tensor data is encoded with a distribution encoding, and
the tensor operation pipeline decodes the distribution encoding.

### Claim 8 (depends on 1)

The computing method of claim 1, wherein
the tensor operation pipeline performs block scaling on the tensor data.

### Claim 9 (depends on 1)

The computing method of claim 1, wherein
the tensor operation pipeline reduces the precision of the tensor data or increases the precision of the tensor data.

### Claim 10 (depends on 1)

The computing method of claim 1, wherein the tensor operation logic units that form the tensor operation pipeline are separate from a tensor arithmetic unit of the hardware accelerator.

### Claim 11 (depends on 10)

The computing method of claim 10, wherein the tensor operation pipeline result is passed to the tensor arithmetic unit for further on-chip processing prior to outputting the tensor operation pipeline result.

### Claim 12

A computing method, comprising:
receiving from a processor a pipeline command including a software-defined tensor operation pipeline definition defining a plurality of tensor operation stages in a tensor operation pipeline and associated predetermined tensor operations to be performed at each of the defined tensor operation stages;
receiving tensor data to be computed by the tensor operation pipeline;
implementing the tensor operation pipeline to perform the tensor operations in each of the tensor operation stages on the tensor data using a plurality of fixed tensor operation logic units, to thereby produce a tensor operation pipeline result for the tensor data; and
outputting the tensor operation pipeline result.

### Claim 13 (depends on 12)

The computing method of claim 12, wherein the tensor data includes numerical parameters of a neural network.

### Claim 14 (depends on 13)

The computing method of claim 13, wherein the numerical parameters of the neural network are floating point values including one or more mantissa bits and one or more exponent bits.

### Claim 15 (depends on 12)

The computing method of claim 12, wherein the predetermined types of tensor operations are selected from the group consisting of split, add, subtract, select, concatenate, and perform a lookup to a lookup table.

### Claim 16 (depends on 15)

The computing method of claim 15, wherein the lookup table is programmable to implement a user-defined function.

### Claim 17 (depends on 16)

The computing method of claim 16, wherein the user-defined function is a decoding function for tensor data that is encoded with distribution encoding, or is a block scaling function.

### Claim 18 (depends on 15)

The computing method of claim 15, wherein the tensor data includes floating point values and the split function splits floating point values into constituent mantissa and exponent portions.

### Claim 19

A computing method, comprising:
receiving a tensor operation pipeline definition and tensor data from a processor, at a configurable pipeline processing element array of a hardware accelerator;
in each of a plurality of processing elements of the array, processing the tensor data by implementing a configurable tensor operation pipeline including one or more of the fixed tensor operation logic units according to the tensor operation pipeline definition, wherein at least one of the fixed tensor operation logic units is a look up table logic unit, values for the look up table being included in the tensor operation pipeline definition, and wherein the stages are in a predetermined order defined by an on-chip hardware layout and identical in each of the processing elements, and individual fixed tensor operation logic units can be turned on or off by commands included in the tensor operation pipeline definition; and
outputting a tensor operation pipeline result based on the processing of the tensor data by each tensor operation pipeline in each processing element.

### Claim 20 (depends on 19)

The method of claim 19, wherein pipeline further includes additional fixed tensor operation logic units selected from consisting of a split logic unit, add logic unit, subtract logic unit, select logic unit, and concatenate logic unit.
