Transpose operations using processing element array
AssigneeAmazon Technologies, Inc.
InventorsHaichen Li, Ron Diamant, Jeffrey T. Huynh, Yu Zhou, Se jong Oh
Life of this patent
Estimating the term…
Abstract
Provided are systems and methods for transposing a tensor using processing element array operations. In some cases, it may be necessary to transpose elements of a tensor to perform a matrix operation. The tensor may be decomposed into blocks of data elements having dimensions consistent with the dimensions of a systolic array. An identity multiplication may be performed on each block of data elements loaded into a systolic array and the multiplication products summed in column partitions of a results buffer. The data elements in the column partitions of results buffer can then be mapped to row partitions of a buffer memory for further processing.
Claims20
A method for transposing a tensor using processing element array operations, the method comprising:
- mapping a block of data elements of the tensor to a number of row partitions of a state buffer of an integrated circuit, each row partition having a number of columns;
- loading the data elements into a processing element array of the integrated circuit in corresponding rows and columns of processing elements;
- performing a series of multiplication operations with an identity matrix such that each column of the identity matrix is sequentially multiplied by each column of data elements in the processing element array;
- after each multiplication operation, summing multiplication products for each column of the processing element array that performs a multiplication operation with a column of the identity matrix;
- storing, in a corresponding column partition of a results buffer, the summed multiplication products for each column of the processing element array that performs a multiplication operation, the results buffer having a same number of column partitions as columns in the processing element array, wherein the summed multiplication products for subsequent multiplication operations are stored in subsequent rows for each corresponding column partition, and wherein the results buffer has a same number of rows as the processing element array;
- and
- loading the summed multiplication products stored in the column partitions in the results buffer to a corresponding number of row partitions in the state buffer.
A computer-implemented method, comprising:
- receiving, by a compiler, a set of operations to be performed by an execution engine;
- determining, by the compiler, that the set of operations includes performing a matrix transpose operation on a tensor;
- converting, by the compiler, the matrix transpose operation into an identity matrix multiplication operation;
- and
- generating, by the compiler, a set of instructions operable to transpose the tensor by loading data elements of the tensor from a buffer memory of the execution engine into a systolic array of the execution engine, performing an identity multiplication on the data elements in the systolic array, storing a result of the identity multiplication in a results buffer of the execution engine, and loading the result from the results buffer into the buffer memory.
A non-transitory computer readable medium having stored therein instructions that, when executed by one or more processors cause the one or more processors to execute a compiler, the compiler performing operations including:
- receiving a set of operations to be performed by an execution engine;
- determining that the set of operations includes performing a matrix transpose operation on a tensor;
- converting the matrix transpose operation into an identity matrix multiplication operation;
- and
- generating a set of instructions operable to transpose the tensor by loading data elements of the tensor from a buffer memory of the execution engine into a systolic array of the execution engine, performing an identity multiplication on the data elements in the systolic array, storing a result of the identity multiplication in a results buffer of the execution engine, and loading the result from the results buffer into the buffer memory.
Dependencies are inferred from the claim text.