Input value cache for temporarily storing input values

A neural network inference circuit that executes a neural network comprising a plurality of computation nodes at a plurality of layers, each computation node of a set of the computation nodes comprising a dot product of input values and weight values, the neural network inference circuit comprising:

• a set of memories to store input values and weight values;
• a memory read controller circuit to (i) read a plurality of input values for a particular layer from a memory location of the set of memories and (ii) store the plurality of input values in a cache of the neural network inference circuit;
• and
• a set of dot product computation circuits to (i) compute a first dot product for a first computation node of the particular layer using a first set of the plurality of input values and (ii) compute a second dot product for a second computation node of the particular layer using a subset of the first set of input values and a second set of the plurality of input values,
• wherein the memory read controller circuit does not read any input values from any additional memory locations between computing the first dot product and computing the second dot product.