Irregular cadence data processing units
AssigneeGoogle Inc.
InventorsIndranil Chakraborty, Rahul Nagarajan, Christopher Aaron Clark
Life of this patent
Estimating the term…
Abstract
Aspects of the disclosure are directed to an architecture including a dynamic serialization buffer and/or dynamic deserialization buffer coupled between a vector processing unit and a matrix multiplication unit. The dynamic serialization buffer and/or dynamic deserialization buffer allow for streaming any integer of vectors per cycle when performing acceleration of matrix multiplication operations. The matrix multiplication unit receives vectors equivalent to an amount of data from the vector processing unit at an arbitrary rate of vectors per cycle. The matrix multiplication unit processes the vectors to generate resulting vectors that are output at the arbitrary rate.
Claims20
An architecture for arbitrary throughput of matrix multiplication, the architecture comprising:
- a dynamic serialization buffer coupled to an input of a matrix multiplication unit and a dynamic deserialization buffer coupled to an output of the matrix multiplication unit;
- the dynamic serialization buffer configured to:
- receive a first amount of data from a first vector register;
- output a second amount of data, the second amount of data being less than the first amount of data, at each of a number of cycles until the dynamic serialization buffer contains an amount of data less than the second amount of data;
- and
- based on containing the amount of data less than the second amount of data, receive a subsequent first amount of data from the first vector register or a second vector register;
- the matrix multiplication unit configured to:
- receive the second amount of data;
- process the second amount of data to generate a third amount of data representing results of the matrix multiplication;
- and
- output the third amount of data;
- and
- the dynamic deserialization buffer configured to:
- receive the third amount of data at each of the number of cycles until the dynamic deserialization buffer contains an amount of data greater than a fourth amount of data, the fourth amount of data being greater than the third amount of data;
- and
- based on containing the amount of data greater than the fourth amount of data, output the fourth amount of data to the first vector register, the second vector register, or a third vector register.
A method for operating an architecture for arbitrary throughput of matrix multiplication, the architecture including a dynamic serialization buffer coupled to an input of a matrix multiplication unit and a dynamic deserialization buffer coupled to an output of the matrix multiplication unit, the method comprising:
- receiving, by the dynamic serialization buffer, a first amount of data from a first vector register;
- outputting, by the dynamic serialization buffer, a second amount of data, the second amount of data being less than the first amount of data, at each of a number of cycles until the dynamic serialization buffer contains an amount of data less than the second amount of data;
- based on containing the amount of data less than the second amount of data, receiving, by the dynamic serialization buffer, a subsequent first amount of data from the first vector register or a second vector register;
- receiving, by the matrix multiplication unit, the second amount of data;
- processing, by the matrix multiplication unit, the second amount of data to generate a third amount of data representing results of the matrix multiplication;
- outputting, by the matrix multiplication unit, the third amount of data;
- receiving, by the dynamic deserialization buffer, the third amount of data at each of the number of cycles until the dynamic deserialization buffer contains an amount of data greater than a fourth amount of data, the fourth amount of data being greater than the third amount of data;
- and
- based on containing the amount of data greater than the fourth amount of data, outputting, by the dynamic deserialization buffer, the fourth amount of data to the first vector register, the second vector register, or a third vector register.
A non-transitory computer readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations for operating an architecture for arbitrary throughput of matrix multiplication, the architecture including a dynamic serialization buffer coupled to an input of a matrix multiplication unit and a dynamic deserialization buffer coupled to an output of the matrix multiplication unit, the operations comprising:
- receiving, by the dynamic serialization buffer, a first amount of data from a first vector register;
- outputting, by the dynamic serialization buffer, a second amount of data, the second amount of data being less than the first amount of data, at each of a number of cycles until the dynamic serialization buffer contains an amount of data less than the second amount of data;
- based on containing the amount of data less than the second amount of data, receiving, by the dynamic serialization buffer, a subsequent first amount of data from the first vector register or a second vector register;
- receiving, by the matrix multiplication unit, the second amount of data;
- processing, by the matrix multiplication unit, the second amount of data to generate a third amount of data representing results of the matrix multiplication;
- outputting, by the matrix multiplication unit, the third amount of data;
- receiving, by the dynamic deserialization buffer, the third amount of data at each of the number of cycles until the dynamic deserialization buffer contains an amount of data greater than a fourth amount of data, the fourth amount of data being greater than the third amount of data;
- and
- based on containing the amount of data greater than the fourth amount of data, outputting, by the dynamic deserialization buffer, the fourth amount of data to the first vector register, the second vector register, or a third vector register.
Dependencies are inferred from the claim text.