Shifting input values within input buffer of neural network inference circuit

A neural network inference circuit that executes a neural network comprising a plurality of computation nodes, each computation node of a set of the computation nodes comprising a dot product of input values and weight values, the neural network inference circuit comprising:

• an input buffer to store a set of input values for a dot product computation;
• a set of dot product computation circuits to compute dot products between the set of input values stored in the input buffer and a plurality of sets of weight values;
• and
• an input buffer configuration circuit to (i) load a first set of input values into the input buffer over a first plurality of clock cycles of the neural network inference circuit and (ii) after a first plurality of dot products for a first plurality of computation nodes of the neural network are simultaneously computed in a first single clock cycle of the neural network inference circuit after the first plurality of clock cycles, wherein each respective dot product of the first plurality of dot products is between the first set of input values and a different respective set of the plurality of sets of weight values, shift the first set of input values in the input buffer while loading a second set of input values into the input buffer over a second plurality of clock cycles of the neural network inference circuit after the first single clock cycle such that (1) a first subset of the first set of input values is removed from the input buffer, (2) a second subset of the first set of input values is moved to new locations in the input buffer, and (3) the second set of input values are loaded into locations in the input buffer vacated by the shifting of the first set of input values,
• wherein the set of dot product computation circuits (i) simultaneously computes the first plurality of dot products for the first plurality of computation nodes in the first single clock cycle and (ii) simultaneously computes a second plurality of dot products for a second plurality of computation nodes of the neural network in a second single clock cycle of the neural network inference circuit after the second plurality of clock cycles, wherein each respective dot product of the second plurality of dot products is between (i) the second set of input values and the second subset of the first set of input values and (ii) a different respective set of the plurality of sets of weight values, wherein the respective sets of weight values used for the first plurality of dot products are the same as the respective sets of weight values used for the second plurality of dot products.