Assigning dnn weights to a 3d crossbar array

A compute-in-memory accelerator system comprising:

• a plurality of in-memory tiles, each in-memory tile comprising more than one tier arranged in the Z dimension, each tier including an array of memory devices adapted for storing a 2D weight-matrix representing at least a portion of a neural network model layer, wherein a mapping of a sequence of at least Ntier1 neural network model layers to tiers of successive in-memory tiles is optimized for a finite input batch-size m of an incoming workflow, said mapping comprising at least
• an assignment of layers Nstart to Nstart+Ntier1−1 of the neural network model to a first tier (tier 1) of successive in-memory tiles 1 to Ntiles, each first tier of the successive in-memory tiles 1 to Ntiles configured for storing data representing a 2D weight-matrix for processing at the corresponding neural network model layer;
• and
• an assignment of layer Nstart+Ntier1 of the neural network model to a second tier (tier 2) of in-memory tile 1, the second tier of in-memory tile 1 configured for storing data representing a 2D weight-matrix for processing at the corresponding neural network model layer,
• wherein said Ntiles is a minimum number of tiles chosen such that the first batch member completes processing in tier 1 of tile Ntiles no sooner than the mth batch member completes processing in tier 1 of in-memory tile 1;
• and
• a controller unit associated with each in-memory tile configured for controlling a 2D weight-matrix multiplication operation of at least a portion of a neural network model layer at a tier of said in-memory tile.

A method for operating a compute-in-memory accelerator system comprising:

• configuring a plurality of in-memory tiles to store data for processing a neural network model, each in-memory tile comprising more than one tier arranged in the Z dimension, each tier comprising an array of memory devices adapted to store a 2D weight-matrix of data representing a neural network model layer;
• mapping of a sequence of greater than N neural network model layers to tiers of successive in-memory tiles optimized for a finite sample batch-size m of an incoming workflow, said mapping comprising at least an assignment of layers Nstart to Nstart+Ntier1−1 of the neural network model to a first tier (tier 1) of successive in-memory tiles 1 to Ntiles, each first tier of the successive in-memory tiles 1 to Ntiles configured for storing data representing a 2D weight-matrix for processing at the corresponding neural network model layer;
• and an assignment of layer Nstart+Ntier1 of the neural network model to a second tier (tier 2) of in-memory tile 1, the second tier of in-memory tile 1 configured for storing data representing a 2D weight-matrix for processing at the corresponding neural network model layer;
• wherein said Ntiles is a minimum number of tiles chosen such that the first batch member completes processing in tier 1 of tile Ntiles no sooner than the mth batch member completes processing in tier 1 of in-memory tile 1;
• and
• controlling a processing of a 2D weight-matrix multiplication operation of at least a portion of a said N neural network model layer at a tier of an in-memory tile.

A compute-in-memory accelerator system comprising:

• a plurality of in-memory tiles, each in-memory tile comprising more than one tier arranged in the Z dimension, each tier including an array of memory devices for storing a 2D weight-matrix of data representing a neural network model layer;
• a mapping of a sequence of greater than N neural network model layers to tiers of successive in-memory tiles optimized for a large sample batch-size m of an incoming workflow, said mapping comprising at least an assignment of a pre-determined amount of successive neural network model layers to respective successive tiers of a single in-memory tile, each successive tier of the single in-memory tile configured for storing data representing a 2D weight-matrix for processing at the corresponding neural network model layer;
• and
• a hardware controller device configured for controlling a 2D weight-matrix multiplication operation at each said successive neural network model layer at each said successive tier of said given in-memory tile.

A method for operating a compute-in-memory accelerator system comprising:

• configuring a plurality of in-memory tiles to store data for processing a neural network model, each in-memory tile comprising more than one tier arranged in the Z dimension, each tier comprising an array of memory devices adapted to store a 2D weight-matrix of data representing a neural network model layer;
• mapping of a sequence of greater than N neural network model layers to tiers of successive in-memory tiles optimized for a large sample batch-size m of an incoming workflow, said mapping comprising assigning a pre-determined amount of successive neural network model layers to respective successive tiers of a single in-memory tile, each successive tier of the single in-memory tile configured for storing data representing a 2D weight-matrix for processing at the corresponding neural network model layer;
• and
• controlling a 2D weight-matrix multiplication operation at each said successive neural network model layers at each said successive tier of said given in-memory tile.