Multiple busses in a grouped systolic array
AssigneeAmazon Technologies, Inc.
InventorThomas A Volpe
Life of this patent
Estimating the term…
Abstract
Systems and methods are provided to enable parallelized multiply-accumulate operations in a systolic array. Each row of the systolic array can include multiple busses enabling independent transmission of inputs along the respective bus. Each processing element of a given row-oriented bus can receive an input from a prior element of the given row-oriented bus, and perform arithmetic operations on the input. The systolic array can be divided into a plurality of sub-arrays corresponding to a row-oriented bus where each sub-array is separated by a shifter. Each shifter can shift a row-oriented bus into the active bus position for a given sub-array. Use of row-oriented busses can enable parallelization to increase speed or enable increased latency at individual processing elements.
Claims26
A systolic processor comprising:
- a systolic array of processing elements arranged in rows and columns, each row of the systolic array including a plurality of row-oriented buses;
- wherein the systolic array of processing elements is divided into a first sub-array of processing elements and a second sub-array of processing elements, each sub-array of processing elements including one or more consecutive columns of the systolic array, each sub-array of processing elements further including, for each row, an active row-oriented bus and one or more inactive row-oriented busses, each processing element configured to perform a mathematical operation on an input of the active row-oriented bus corresponding to the respective sub-array;
- a first shifter element interposed between the first and second sub-arrays, wherein the first shifter element comprises circuitry configured, for each row of the array, to shift a data element transmitted on an inactive bus of the first sub-array to an active bus of the second sub-array;
- wherein each processing element is further configured to:
- perform the mathematical operation on a weight and an input data element, and
- provide the weight and the input data element to a subsequent column of the respective sub-array.
A systolic processor comprising:
- a systolic array of processing elements arranged in rows and columns;
- wherein the systolic array of processing elements is divided into a plurality of sub-arrays of processing elements, each sub-array of the processing elements including a plurality of consecutive columns of the systolic array, each sub-array of the processing elements further including, for each row, an active bus and one or more inactive busses, each processing element of a sub-array configured to perform a mathematical operation on an input of the active bus corresponding to the respective sub-array;
- and
- a plurality of shifter elements dispersed within the systolic array, each shifter element comprising circuitry configured, for each row of the array, to obtain an output and provide the output to an active bus of a sub-array.
A systolic processor comprising:
- a systolic array of processing elements arranged in rows and columns, each row of the systolic array including a plurality of row-oriented busses, each row-oriented bus corresponding to a number of consecutive processing elements;
- wherein each row-oriented bus is configured as an active row-oriented bus for a subset of consecutive columns of the systolic array, each processing element of the subset of consecutive columns of the systolic array configured to perform a mathematical operation on an input of the active row-oriented bus;
- and
- a plurality of shifter elements, each shifter element configured to select a row-oriented bus for a given subset of consecutive columns, wherein selecting the row-oriented bus comprises selecting the row-oriented bus for the active row-oriented bus of the given subset of consecutive columns, each shifter element interposed within the systolic array.
Dependencies are inferred from the claim text.