System and method for increasing utilization of dot-product based neural network accelerator

A method of accelerating convolution of an input feature map in an inference system comprising a dot-product circuit, the method comprising:

• loading kernel weights corresponding to a channel of the plurality of channels of the input feature map into a kernel weight vector of the dot-product circuit;
• flattening channel data of the input feature map by:
• retrieving pixel values of a channel of a plurality of channels of the input feature map from a memory and storing the pixel values in a buffer;
• extracting first values of a first region having a first size from among the pixel values stored in the buffer, the first region corresponding to an overlap region of a kernel of the inference system with channel data of the input feature map;
• rearranging, in a row-major or a column-major order, second values corresponding to the overlap region of the kernel from among the first values in the first region into a single one-dimensional vector;
• identifying a first group of consecutive values from among the rearranged second values of the single one-dimensional vector for supplying to a first dot-product circuit of the inference system;
• and
• identifying a second group of consecutive values from among the rearranged second values of the single one-dimensional vector for supplying to a second dot-product circuit of the inference system;
• loading the first group of consecutive values into a first activation vector of the first dot-product circuit and the second group of consecutive values into a second activation vector of the second dot-product circuit;
• and
• calculating an inner product of the first and second activation vectors and the kernel weight vector to generate an output feature map.

An image flattening circuit for flattening channel data of an input feature map in an inference system, the flattener comprising:

• a channel controller configured to retrieve pixel values of a channel of a plurality of channels of the input feature map from a memory and storing the pixel values in a buffer;
• a max sub-window isolator configured to extract first values of a first region having a first size from among the pixel values stored in the buffer, the first region corresponding to an overlap region of a kernel of the inference system with channel data of the input feature map;
• and
• a rearranging circuit configured to rearrange, in a row-major or a column-major order, second values corresponding to the overlap region of the kernel from among the first values in the first region into a single one-dimensional vector, to identify a first group of consecutive values from among the rearranged second values of the single one-dimensional vector for supplying to a first dot-product circuit of the inference system, and to identify a second group of consecutive values from among the rearranged second values of the single one-dimensional vector for supplying to a second dot-product circuit of the inference system.