Accelerator Architecture For A Transformer Machine Learning Model

An accelerator architecture for a Transformer machine learning model, comprising:

• one or more memory devices, each memory device has a random access memory and is configured for processing in memory, where a key matrix, a value matrix, a query weight matrix, a key weight matrix, and a value weight matrix for an attention mechanism of the Transformer machine learning model are stored in the one or more memory devices;
• and
• a control device interfaced with each of the one or more memory devices, the control device is configured to receive an input vector and coordinates a first vector-matrix multiplication of the input vector with the query weight matrix, such that the first vector-matrix multiplication operation is performed on the one or more memory devices and an output of the first vector-matrix multiplication operation is used by the control device for subsequent processing;
• the control device coordinates a second vector-matrix multiplication operation between the input vector and the key weight matrix, such that the second vector-matrix multiplication operation is performed on the one or more memory devices and an output of the second vector-matrix multiplication operation is used by the control device for subsequent processing;
• the control device coordinates a third vector-matrix multiplication operation between the input vector and the value weight matrix, such that the third vector-matrix multiplication operation is performed on the one or more memory devices and an output of the third vector-matrix multiplication operation is used by the control device for subsequent processing.