Programmable compute engine having transpose operations
AssigneeAmazon Technologies, Inc.
InventorsXiaodan Tan, Paul Gilbert Meyer, Sheng Xu, Ron Diamant
Life of this patent
Estimating the term…
Abstract
A technique to execute transpose and compute operations may include retrieving a set of machine instructions from an instruction buffer of a data processor. The instruction buffer has multiple entries, and each entry stores one machine instruction. A machine instruction from the set of machine instructions is executed to transpose a submatrix of an input tensor and perform computations on column elements of the submatrix. The machine instruction combines the transpose operation with computational operations into a single machine instruction.
Claims20
A computer-implemented method comprising:
- receiving, by a data processor, a set of machine instructions to execute, wherein the set of machine instructions implements a neural network operation to normalize data elements along columns of an input tensor;
- and
- executing, by the data processor, the set of machine instructions including:
- executing a transpose-and-compute machine instruction to transpose submatrices of the input tensor in a transpose circuit of a vector compute bank, compute statistical values on column elements of the submatrices being streamed into compute channels of the vector compute bank, and store the statistical values in a set of row partitions of a memory for the vector compute bank;
- and
- executing a scaling machine instruction to scale data elements of the input tensor being streamed into the compute channels of the vector compute bank based on the statistical values.
A computer-implemented method comprising:
- retrieving a set of machine instructions from an instruction buffer of a data processor, wherein the instruction buffer includes a plurality of entries, and wherein each entry stores one machine instruction;
- and
- executing a first machine instruction from the set of machine instructions to transpose a first submatrix of an input tensor and perform first computations on column elements of the first submatrix,
- wherein the first machine instruction combines a transpose operation with computational operations into a single machine instruction, and
- wherein the first machine instruction is executed concurrently on multiple vector compute banks to operate on different submatrices of the input tensor in parallel.
A data processor comprising:
- an instruction buffer operable to store pending machine instructions;
- an instruction decoder operable to decode machine instructions stored in the instruction buffer for execution;
- and
- a processing circuit having a plurality of compute channels and operable to execute a machine instruction to transpose a submatrix of an input tensor and perform computations on column elements of the submatrix using respective compute channels,
- wherein the machine instruction combines a transpose operation with computational operations including computing a mean or a variance of each column of the submatrix into a single machine instruction.
A non-transitory computer readable medium having stored therein instructions that, when executed by one or more processors, cause the one or more processors to execute a compiler, the compiler performing operations comprising:
- receiving a description of a neural network model that contains neural network operations;
- identifying a neural network operation in the description of the neural network model that performs computations including computing a mean or a variance on data elements of vectors that are each stored across row partitions of a memory buffer by a previous neural network operation;
- and
- generating a machine instruction that combines a transpose operation with computational operations including computing the mean or the variance to implement the identified neural network operation.
Dependencies are inferred from the claim text.