Decreased quantization latency

A method of decreasing quantization latency, the method comprising:

• determining a first integer data type of data at least one layer of a neural network is configured to process;
• determining a second integer data type of data received for processing by the neural network, the second integer data type being different than the first integer data type;
• determining a ratio between a first size of the first integer data type and a second size of the second integer data type;
• scaling parameters of the at least one layer of the neural network using a scaling factor corresponding to the ratio;
• quantizing the scaled parameters of the neural network;
• and
• inputting the received data to the neural network with the quantized and scaled parameters.

A non-transitory computer-readable medium having stored thereon instructions that, when executed by one or more processors, cause the one or more processors to:

• determine a first integer data type of data at least one layer of a neural network is configured to process;
• determine a second integer data type of data received for processing by the neural network, the second integer data type being different than the first integer data type;
• determine a ratio between a first size of the first integer data type and a second size of the second integer data type;
• scale parameters of the at least one layer of the neural network using a scaling factor corresponding to the ratio;
• quantize the scaled parameters of the neural network;
• and
• input the received data to the neural network with the quantized and scaled parameters.