Architecture for ai accelerator platform

A system, comprising:

• a plurality of compute tiles, each of the plurality of compute tiles including a plurality of compute engines and a general-purpose (GP) processor coupled with the plurality of compute engines, each of the plurality of compute engines including a compute-in-memory (CIM) hardware module, the CIM hardware module including fully addressable memory cells and hardware compute logic coupled with the fully addressable memory cells, the CIM hardware module storing a plurality of weights corresponding to a matrix in the fully addressable memory cells, and the hardware compute logic being configured to perform a vector-matrix multiplication (VMM) for the matrix, the GP processor being configured to provide control instructions and data to the plurality of compute engines, the GP processor being a reduced instruction set computer (RISC) processor;
• wherein each of the plurality of compute engines is individual coupled with the GP processor such that each of the plurality of compute engines is configured to store data from the GP processor based on a load instruction from the GP processor, perform the VMM, and store output to the GP processor based on to a store instruction from the GP processor, wherein at least a first portion of the VMM for each of the plurality of compute engines is performable in parallel with at least a second portion of the VMM for another of the plurality of compute engines.

A method, comprising:

• providing, to at least one compute engine of a plurality of compute engines, an input vector, the at least one compute engine storing a plurality of weights corresponding to a matrix, each of the plurality of compute engines including a compute-in-memory (CIM) hardware module, the CIM hardware module including fully addressable memory cells and hardware compute logic coupled with the fully addressable memory cells, the CIM hardware module of the at least one compute engine storing the plurality of weights in the fully addressable memory cells, and the hardware compute logic being configured to perform a vector-matrix multiplication (VMM) for the matrix, the at least one compute engine performing a VMM between the input vector and the matrix to provide an output;
• applying, by a general-purpose (GP) processor coupled with the plurality of compute engines, a function to the output, the GP processor being configured to provide control instructions and data to the plurality of compute engines, the GP processor being a reduced instruction set computer (RISC) processor;
• wherein each of the plurality of compute engines is coupled with the GP processor such that each of the plurality of compute engines is configured to store data from the GP processor based on a load instruction from the GP processor, perform the VMM, and store output to the GP processor based on a store instruction from the GP processor, wherein at least a first portion of the VMM for each of the plurality of compute engines is performable in parallel with at least a second portion of the VMM for another of the plurality of compute engines.