Neural network circuit using affine transforms to encode activations

A neural network inference circuit comprising:

• a set of core computation circuits configured to:
• generate, using a set of input values for a neural network, a first value, wherein the first value is output of a core computation for a first node of the neural network;
• a set of affine transform circuits configured to:
• generate, based on the first value, a second value, wherein generating the first value comprises:
• applying a first affine transform to the second value, wherein the first affine transform is a combination of (i) a batch normalization transform, (ii) a learned affine transform, (iii) a weight scale for the first node, and (iv) an encoding transform associated with a first activation function such that the second value is encoded for input in a first input range associated with the first activation function;
• and
• a set of activation circuits configured to apply the first activation function to the second value to generate an output value for the first node.