Optimizing low precision inference models for deployment of deep neural networks

A semiconductor apparatus comprising:

• one or more substrates;
• and
• logic coupled to the one or more substrates, wherein the logic is implemented at least partly in one or more of configurable logic or fixed-functionality hardware logic, the logic coupled to the one or more substrates to:
• generate a quantized neural network, wherein model weights of the neural network are symmetrically quantized as signed integer values, and wherein an input layer of the neural network is configured to asymmetrically quantize input values as unsigned integer values;
• generate a weights accumulation table based on the quantized model weights and a kernel size for the neural network, wherein the weights accumulation table bis an accumulation of integer weights within a range of kernel indices;
• and
• generate an output restoration function for an output layer of the neural network based on the weights accumulation table, convolution result, and the kernel size.

At least one non-transitory computer readable storage medium comprising a set of instructions for managing a runtime computing environment which, when executed by a computing system, cause the computing system to:

• generate a quantized neural network, wherein model weights of the neural network are symmetrically quantized as signed integer values, and wherein an input layer of the neural network is configured to quantize input values as asymmetrically unsigned integer values;
• generate a weights accumulation table based on the quantized model weights and a kernel size for the neural network, wherein the weights accumulation table bis an accumulation of integer weights within a range of kernel indices;
• and
• generate an output restoration function for an output layer of the neural network based on the weights accumulation table, convolution result, and the kernel size.

A method comprising:

• generating a quantized neural network, wherein model weights of the neural network are symmetrically quantized as signed integer values, and wherein an input layer of the neural network is configured to quantize input values as asymmetrically unsigned integer values;
• generating a weights accumulation table based on the quantized model weights and a kernel size for the neural network, wherein the weights accumulation table bis an accumulation of integer weights within a range of kernel indices;
• and
• generating an output restoration function for an output layer of the neural network based on the weights accumulation table, convolution result, and the kernel size.