Convolutional neural network hardware configuration

A computer system for configuring a hardware implementation of a Convolutional Neural Network (CNN), the computer system comprising:

• hardware logic configured to determine, for each of a plurality of layers of the CNN, a first number format for representing weight values in the layer based upon a distribution of weight values for the layer, the first number format comprising a first integer of a first predetermined bit-length and a first exponent value that is fixed for the layer;
• and
• hardware logic configured to determine, for each of a plurality of layers of the CNN, a second number format for representing data values in the layer based upon a distribution of expected data values for the layer, the second number format comprising a second integer of a second predetermined bit-length and a second exponent value that is fixed for the layer;
• said computer system further configured to store the determined number formats in a computer readable storage medium for use in configuring the hardware implementation of a CNN;
• wherein determining the first exponent value for each layer comprises:
• identifying a bit index that represents a range of values based upon the distribution of weight values for that layer,
• right-shifting the position of the bit index from the identified bit index a number of bits based upon the first predetermined bit-length to identify a first shifted bit index, and
• allocating the value of the first shifted bit index as the first exponent value;
• and
• using the stored determined number formats to configure the hardware implementation of the CNN.

A non-transitory computer readable storage medium having stored thereon computer executable instructions that when executed cause at least one processor to configure a hardware implementation of a Convolutional Neural Network (CNN), by:

• determining, for each of a plurality of layers of the CNN, a first number format for representing weight values in the layer based upon a distribution of weight values for the layer, the first number format comprising a first integer of a first predetermined bit-length and a first exponent value that is fixed for the layer;
• determining, for each of a plurality of layers of the CNN, a second number format for representing data values in the layer based upon a distribution of expected data values for the layer, the second number format comprising a second integer of a second predetermined bit-length and a second exponent value that is fixed for the layer;
• and
• storing the determined number formats in a computer readable storage medium for use in configuring the hardware implementation of a CNN;
• wherein determining the first exponent value for each layer comprises:
• identifying a bit index that represents a range of values based upon the distribution of weight values for that layer,
• right-shifting the position of the bit index from the identified bit index a number of bits based upon the first predetermined bit-length to identify a first shifted bit index, and
• allocating the value of the first shifted bit index as the first exponent value;
• and
• using the stored determined number formats to configure the hardware implementation of the CNN.