Convolution hardware accelerator

A system, comprising:

• a plurality of multiplication and accumulation (MAC) cells;
• a feature processor circuit;
• a weight processor circuit;
• and
• a controller circuit configured to:
• for each iteration of an outer loop, generate one or more requests for a different plurality of weight units of a kernel, wherein for each iteration of the outer loop, the weight processor circuit is configured to:
• receive the different plurality of weight units each comprising a plurality of weight elements from different respective channels of a kernel;
• generate a plurality of extended weight units each comprising a weight element from each of the plurality of weight units and from a common channel of the kernel;
• and
• provide the plurality of extended weight units to respective MAC cells of the plurality of MAC cells;
• for each iteration of an inner loop within the outer loop, generate one or more requests for a different plurality of input units of an input tensor, wherein for each iteration of the inner loop, the feature processor circuit is configured to:
• receive the different plurality of input units each comprising a plurality of input feature elements from different respective channels of an input tensor;
• generate a plurality of extended feature units each comprising an input feature element from each of the plurality of input units and from a common channel of the input tensor;
• and
• provide the plurality of extended feature units to respective MAC cells of the plurality of MAC cells, wherein each MAC cell of the plurality of MAC cells is configured to multiply the input feature elements of the extended feature unit provided by the feature processor circuit by the respective weight elements of the extended weight unit provided by the weight processor circuit in parallel and output a sum of the products;
• and
• an accumulator circuit configured to accumulate the sums of the products output by the plurality of MAC cells for each iteration of the inner loop to generate an output tensor representing a depthwise convolution of the input tensor and the kernel;
• wherein the feature processor circuit comprises:
• a first input buffer configured to receive the input units from a memory;
• a first series of delay registers, each delay register having capacity to store one of the input units;
• and
• a first plurality of multiplexer circuits corresponding to respective channels of the input tensor, wherein for each iteration of the inner loop, the feature processor circuit is further configured to:
• write a next input unit from the first input buffer into a first delay register in the first series of delay registers;
• and
• shift input units currently written in a respective delay register in the first series of delay registers to a next delay register in the first series of delay registers;
• wherein each multiplexer circuit is configured to provide a respective extended feature unit comprising a number of input feature elements from the first series of delay registers to a respective MAC cell of the plurality of MAC cells.

A method, comprising:

• for each iteration of an outer loop:
• generating one or more requests for a different plurality of weight units of a kernel;
• receiving the different plurality of weight units each comprising a plurality of weight elements from different respective channels of a kernel;
• generating a plurality of extended weight units each comprising a weight element from each of the plurality of weight units and from a common channel of the kernel;
• and
• providing the plurality of extended weight units to respective multiplication and accumulation (MAC) cells of a plurality of MAC cells;
• for each iteration of an inner loop within the outer loop:
• generating one or more requests for a different plurality of input units of an input tensor;
• receiving the different plurality of input units each comprising a plurality of input feature elements from different respective channels of an input tensor;
• generating a plurality of extended feature units each comprising an input feature element from each of the plurality of input units and from a common channel of the input tensor;
• providing the plurality of extended feature units to respective MAC cells of the plurality of MAC cells;
• and
• multiplying, by each MAC cell, the input feature elements of the extended feature unit by the respective weight elements of the extended weight unit in parallel and outputting a sum of the products;
• accumulating the sums of the products output by the plurality of MAC cells for each iteration of the inner loop to generate an output tensor representing a depthwise convolution of the input tensor and the kernel;
• and
• for a first iteration of the inner loop, repeating the steps of:
• writing a next input unit from a first input buffer into a first delay register in a first series of delay registers;
• and
• shifting input units currently written in the first series of delay registers a number of times corresponding to a number of input feature elements included in an extended feature unit before providing the extended feature units to the plurality of MAC cells.