Low overhead implementation of Winograd for CNN with 3x3, 1x3 and 3x1 filters on weight station dot-product based CNN accelerators

A method for image processing to form an output feature map (OFM), the method comprising:

• selecting, by a pipelined structure of a device, activation values in an input feature map (IFM) in each of a predetermined number of input channels of a neural network configured within the device, the activation values being stored in a first memory of the device, each IFM comprising a first predetermined size, and a position in the IFM of each selected activation value being a same position as selected activation values in other IFMs from which the other selected activation values are selected;
• transforming on-the-fly the selected activation values to a Winograd domain by the pipelined structure, wherein the activation values are transformed in a first phase transformation and a second phase transformation, the device including at least one intermediate register to store intermediate results of the first phase transformation and the second phase transformation;
• selecting, by the pipelined structure, an element in a Winograd filter in each of the predetermined number of input channels, each Winograd filter being stored in a second memory of the device and each Winograd filter corresponding to a kernel filter comprising the first predetermined size, each selected element in a Winograd filter corresponding to an activation value in the Winograd domain, and a position of each respective element in the Winograd filter from which the element is selected being a same position as another selected element in the Winograd filter from which the another element is selected;
• multiplying, by a multiplier of the device, a transformed activation value in the Winograd domain by a corresponding element in a Winograd filter to form a corresponding product value in the Winograd domain;
• repeatedly selecting, by the pipelined structure, activation values from an IFM in each of the predetermined number of input channels stored in the first memory, transforming the selected activation values on-the-fly to the Winograd domain, selecting an element in a Winograd filter stored in the second memory that corresponds to a currently transformed activation value in the Winograd domain, and multiplying, by the multiplier, the currently transformed activation value and the corresponding element to form a corresponding product value in the Winograd domain until all activation values of the IFM have been transformed and multiplied by the corresponding element;
• summing, by an adder of the device, the respective product values in the Winograd domain to form elements of a feature map in the Winograd domain;
• and
• inverse-Winograd transforming on-the-fly, by an inverse-Winograd transformer of an image processing unit of the device, the elements of the feature map in the Winograd domain to an original domain of the activation values to form an OFM, the OFM comprising a second predetermined size;
• wherein the activation values are transformed in a first phase transformation of the pipelined structure and a second phase transformation of the pipelined structure, the device including at least one intermediate register to store intermediate results of the first phase transformation and the second phase transformation.