US 11,204,889 B1Granted

Tensor partitioning and partition access order

AssigneeSambaNova Systems, Inc.

InventorsRaghu Prabhakar, Nathan Francis Sheeley, Matheen Musaddiq, Scott Layson Burson, Sitanshu Gupta, Sumti Jairath, Pramod Nataraja, Ajit Punj

Life of this patent

Estimating the term…

patentgrep

Loading patent…

Abstract

A method of processing partitions of a tensor in a target order includes receiving, by a reorder unit and from two or more producer units, a plurality of partitions of a tensor in a first order that is different from the target order, storing the plurality of partitions in the reorder unit, and providing, from the reorder unit, the plurality of partitions in the target order to one or more consumer units. In an example, the one or more consumer units process the plurality of partitions in the target order.

Claims20

Dependent claims

A method of processing partitions of a tensor in a target order, comprising:

receiving, by a reorder unit of an array of configurable units and from two or more producer units of the array of configurable units, a first plurality of partitions of a tensor in a first order,
wherein the tensor comprises (i) the first plurality of partitions and (ii) a second plurality of partitions,
wherein the target order is a concatenation of (i) a first target order that indicates a corresponding desired order of the first plurality of partitions and (ii) a second target order that indicates a corresponding desired order of the second plurality of partitions,
wherein the first order in which the first plurality of partitions of the tensor is received is different from the first target order;
storing the first plurality of partitions in the reorder unit;
providing, from the reorder unit, the first plurality of partitions in the first target order to one or more consumer units of the array of configurable units;
subsequent to and in response to providing the first plurality of partitions in the first target order to the one or more consumer units, issuing, by the reorder unit, a reset signal,
wherein no partition from the second plurality of partitions of the tensor are transmitted to the reorder unit prior to issuance of the reset signal, and no partition from the first plurality of partitions of the tensor are transmitted to the reorder unit subsequent to issuance of the reset signal;
subsequent to and in response to issuing the reset signal, receiving, by the reorder unit and from the two or more producer units, the second plurality of partitions of the tensor in a second order that is different from the second target order;
storing the second plurality of partitions in the reorder unit;
providing, from the reorder unit, the second plurality of partitions in the second target order to the one or more consumer units;
and
processing, by the one or more consumer units, the first and second plurality of partitions in the target order.

10 more elements

An integrated circuit, comprising:

an array of configurable units, configurable units in the array of configurable units including a plurality of producer configurable units, a reorder configurable unit, and one or more consumer configurable units;
and
a data bus connected to the configurable units which communicates data at a data bus clock rate,
wherein a tensor is partitioned in at least a first plurality of partitions and a second plurality of partitions,
wherein a combination of a first producer configurable unit and a second producer configurable unit of the plurality of producer configurable units is configured to store the first and second plurality of partitions of the tensor,
wherein the consumer configurable units are to require transmission of (i) the first plurality of partitions to the consumer configurable units in a first target order for processing, and (ii) the second plurality of partitions to the consumer configurable units in a second target order for processing,
wherein the plurality of producer configurable units is configured to deliver the first plurality of partitions to the reorder configurable unit in an order that is different from the first target order, and are configured to provide to the reorder configurable unit first ordering information identifying the first target order,
wherein the reorder configurable unit is configured to use the first ordering information to reorder the first plurality of partitions in the first target order, and to deliver the reordered first plurality of partitions in the first target order to the consumer configurable units,
wherein the reorder configurable unit is configured to issue a read ready signal, in response to transmission of the reordered first plurality of partitions to the consumer configurable units,
wherein the plurality of producer configurable units is configured to, in response to the read ready signal, deliver the second plurality of partitions to the reorder configurable unit in another order that is different from the second target order, and are configured to provide to the reorder configurable unit second ordering information identifying the second target order,
wherein the reorder configurable unit is configured to use the second ordering information to reorder the second plurality of partitions in the second target order, and to deliver the reordered second plurality of partitions in the second target order to the consumer configurable units, and
wherein the consumer configurable units are configured to process the reordered first and second plurality of partitions in a target order that is a concatenation of the first target order and the second target order.

9 more elements

An integrated circuit, comprising:

an array of configurable units, configurable units in the array of configurable units including processing units and reorder configurable units;
a data bus connected to the array of configurable units which communicates data at a data bus clock rate;
the array of configurable units configured to partition a tensor into a plurality of subtensors, to distribute processing of subtensors in the plurality of subtensors across a set of the processing units at the data bus clock rate, and to require outputs generated from processing the subtensors to be in a target order that is a concatenation of a first target order and a second target order;
processing units in the set of processing units that process the subtensors and generate the outputs, and are configured to provide to the reorder configurable units ordering information identifying the first and second target orders, wherein the outputs comprise a first plurality of outputs and a second plurality of outputs, wherein the processing units in the set of processing units are configured to (i) deliver the first plurality of outputs to the reorder configurable units in an order that is different from the first target order, (ii) receive a reset signal subsequent to delivering the first plurality of outputs to the reorder configurable units, and (iii) subsequent to receiving the reset signal, deliver the second plurality of outputs to the reorder configurable units in another order that is different from the second target order;
the reorder configurable units configured to use the ordering information to (i) reorder the first plurality of outputs in the first target order, and deliver the reordered first plurality of outputs to another set of the processing units, and (ii) reorder the second plurality of outputs in the second target order, and deliver the reordered second plurality of outputs to the another set of the processing units;
and
processing units in the another set of processing units configured to process the reordered first and second plurality of outputs in the target order.

4 more elements

Dependencies are inferred from the claim text.

Facts

Application: 17/216,647Filed 29 Mar 2021
Granted: 21 Dec 2021
Term ends: Not estimatedThe source records don’t support an estimate.

Classification8 codes

G06F 15/7871
With reconfigurable architecture: reconfiguration support, e.g. configuration loading, configuration switching, or hardware OS
G06F 15/7892
Runtime interface, e.g. data exchange, runtime control: reconfigurable logic embedded in CPU, e.g. reconfigurable unit
G06F 15/7817
System on chip, i.e. computer system on a single chip; System in package, i.e. computer system on one or more chips in a single package: specially…
G06F 15/7839
Comprising a single central processing unit: with memory