Methods and hardware for inter-layer data format conversion in neural networks

A non-transitory computer-readable medium comprising machine-readable code which, when executed by a processor of a neural network hardware accelerator, causes the processor to perform a method comprising:

• executing a neural network, the neural network comprising at least two computation layers including a first layer to process first data in a first data format and a second layer to process second data in a second data format, the second data format differing from the first data format;
• extracting data statistics from data output by the first layer, said data statistics being representative of the data output by the first layer;
• determining one or more conversion parameters based on the extracted data statistics and the second data format;
• generating the second data for the second layer by modifying said data output by the first layer using the one or more conversion parameters;
• and
• providing the generated second data to the second layer for processing.

Hardware for executing a neural network, the neural network comprising at least two computation layers including a first layer to process first data in a first data format and a second layer to process second data in a second data format, the second data format differing from the first data format, the hardware further comprising:

• statistics extraction circuitry configured for extracting data statistics from data output by the first layer, said data statistics being representative of the data output by the first layer;
• format deduction circuitry configured for determining one or more conversion parameters based on the extracted data statistics and the second data format;
• and
• modification circuitry configured for generating the second data for the second layer by modifying said data output by the first layer using the one or more conversion parameters and for providing the generated second data to the second layer for processing.

A method of quantization-aware training of a neural network executing on a hardware accelerator, the neural network comprising at least two computation layers including a first layer to process first data in a first data format and a second layer to process second data in a second data format, the second data format differing from the first data format, the method comprising:

• extracting, by the hardware accelerator, data statistics from data output by the first layer, said data statistics being representative of the data output by the first layer;
• determining, by the hardware accelerator, one or more conversion parameters based on the extracted data statistics and the second data format;
• generating, by the hardware accelerator, the second data for the second layer by modifying said data output by the first layer using the one or more conversion parameters;
• and
• providing, by the hardware accelerator, the generated second data to the second layer for processing.