Compiling asymmetrically-quantized neural network models for deep learning acceleration

A system for neural network computing, comprising:

• a memory to store a compiler and one or more neural network models;
• and
• processing hardware coupled to the memory, the processing hardware operative to:
• receive a neural network model that includes asymmetric operations, each asymmetric operation including one or more fixed-point operands that are asymmetrically-quantized from corresponding floating-point operands;
• compile, by the compiler, a given asymmetric operation of the neural network model into a symmetric operation that includes a combined bias value, wherein the combined bias value is a constant computed by the compiler by merging at least zero points of input and output of the given asymmetric operation;
• and
• generate a symmetric neural network model including the symmetric operation for an accelerator circuit to execute in fixed-point arithmetic.