Batch size pipelined PIM accelerator for vision inference on multiple images

A system for pipelining inference of a neural network comprising a plurality of layers comprising an i-th layer (i being an integer greater than zero), an (i+1)-th layer, and an (i+2)-th layer, the system comprising:

• a processor;
• and
• a processor memory local to the processor, wherein the processor memory has stored thereon instructions that, when executed by the processor, cause the processor to perform:
• processing, for a first input image, by a controller using a composite filter comprising a first i-th filter associated with the i-th layer and a second i-th filter associated with the i-th layer, first i-th values of the i-th layer to generate first (i+1)-th values for the (i+1)-th layer, wherein the first i-th filter and the second i-th filter are offset in position by a stride length of the first i-th filter, such that a stride of the composite filter is greater than or equal to a sum of a stride of the first i-th filter and a stride of the second i-th filter, the stride length of the first i-th filter being associated with a movement of the first i-th filter across the i-th layer;
• processing, for the first input image, the first (i+1)-th values of the (i+1)-th layer to generate output values;
• and
• concurrently with processing, for the first input image, the (i+1)-th values,
• processing, for a second input image, second i-th values of the i-th layer to generate second (i+1)-th values.

A configurable processing-in-memory (PIM) system configured to implement a neural network, the system comprising:

• a first at least one PIM subarray configured to perform a filtering operation of a composite filter comprising a first i-th filter and a second i-th filter of an i-th layer of the neural network (i being an integer greater than zero), wherein the first i-th filter and the second i-th filter are offset in position by a stride length of the first i-th filter, such that a stride of the composite filter is greater than or equal to a sum of a stride of the first i-th filter and a stride of the second i-th filter, the stride length of the first i-th filter being associated with a movement of the first i-th filter across the i-th layer;
• a second at least one PIM subarray configured to perform a filtering operation of a (i+1)-th filter of an (i+1)-th layer of the neural network;
• and
• a controller configured to control the first and second at least one PIM subarrays, the controller being configured to perform:
• supplying first i-th values of the i-th layer to the first at least one PIM subarray to generate first (i+1)-th values for the (i+1)-th layer, the first i-th values corresponding to a first input image;
• supplying the first (i+1)-th values of the (i+1)-th layer to the second at least one PIM subarray to generate output values associated with the first input image;
• and
• concurrently with supplying the (i+1)-th values corresponding to the first input image,
• supplying second i-th values of the i-th layer to the first at least one PIM subarray to generate second (i+1)-th values, the second i-th values corresponding to a second input image.