System and method for processing convolutions on crossbar-based neural network accelerators for increased inference throughput

A computer-implemented method for computing an output for a convolutional neural network, the method comprising:

• for a convolution layer in a plurality of convolution layers configured to perform intra- crossbar parallelization and inter-crossbar parallelization that compute simultaneously on input data points by:
• flattening kernels in the convolution layer into vectors;
• grouping the vectors into a vector matrix, where the vector matrix comprises a plurality of lines;
• replicating and storing duplicates of the vector matrix according to a number and size on the convolution layer of the convolutional neural network and a crossbar size of a crossbar, wherein the duplicates are stored in unused space of the crossbar of an integrated chip, comprising a crossbar matrix;
• and
• computing a first convolution of the convolution layer as a dot product of input activations vector and the crossbar matrix, wherein the first convolution of the convolution layer corresponds with smaller weights than a second convolution of the convolution layer to parallelize separate iterations in the convolution layer in alignment with the intra-crossbar parallelization, and wherein the duplicates are computed simultaneously on the input data points to perform the intra-crossbar parallelization and the inter-crossbar parallelization.

A non-transitory computer-readable storage medium storing a plurality of instructions executable by one or more processors, the plurality of instructions when executed by the one or more processors cause the one or more processors to perform intra-crossbar parallelization and inter-crossbar parallelization that compute simultaneously on input data points by:

• flattening kernels in a convolution layer into vectors;
• grouping the vectors into a vector matrix, where the vector matrix comprises a plurality of lines;
• replicating and storing duplicates of the vector matrix according to a number and size on the convolution layer of a convolutional neural network and a crossbar size of a crossbar, wherein the duplicates are stored in unused space of the crossbar of an integrated chip, comprising a crossbar matrix;
• and
• computing a first convolution of the convolution layer as a dot product of input activations vector and the crossbar matrix, wherein the first convolution of the convolution layer corresponds with smaller weights than a second convolution of the convolution layer to parallelize separate iterations in the convolution layer in alignment with the intra-crossbar parallelization, and wherein the duplicates are computed simultaneously on the input data points to perform the intra-crossbar parallelization and the inter-crossbar parallelization.