Method and apparatus for quantizing parameters of neural network

A method of quantizing parameters of a neural network, the method comprising:

• acquiring a parameter of a floating-point format used in a process of inferring by the neural network;
• calculating an Lp norm of a weight included in the parameter;
• inferring an Lp norm of an output activation of one or more layers configuring the neural network included in the parameter;
• and
• calculating an Lp norm of input data of the neural network.

A neural network quantization apparatus for quantizing parameters of a neural network, the apparatus comprising:

• a processor configured to:
• acquire a parameter of a floating-point format used in a process of inferring by the neural network,
• quantize, based on statistics of a weight included in the parameter, the weight into a fixed-point format,
• determine, based on statistics of an activation of one or more layers configuring the neural network included in the parameter, a dynamic range of the activation, and
• quantize, based on statistics of input data of the neural network, the input data into a fixed-point format.

A neural network quantization apparatus for quantizing parameters of a neural network, the apparatus comprising:

• a processor configured to:
• acquire a parameter of a floating-point format used in a process of inferring by the neural network,
• calculate an Lp norm of a weight included in the parameter,
• infer an Lp norm of an output activation of one or more layers configuring the neural network included in the parameter, and
• calculate an Lp norm of input data of the neural network.

A neural processing unit (NPU) comprising:

• a memory comprising an Lp norm of a weight, an Lp norm of an output activation of layers, and an Lp norm of input data;
• and
• a neural network quantization apparatus configured to:
• determine a dynamic range of the weight based on the Lp norm of the weight,
• quantize the weight into a fixed-point format based on the dynamic range of the weight,
• determine a dynamic range of the output activation of the layers based on the Lp norm of the output activation of the layers,
• determine a dynamic range of the input data based on the Lp norm of the input data, and
• quantize the input data into a fixed-point format based on the dynamic range of the input data.

A neural network inferring apparatus comprising:

• a memory comprising a parameter of a floating-point format used in a process of inferring by a neural network;
• a neural network quantization apparatus configured to acquire the parameter from the memory, quantize, based on statistics of a weight included in the parameter, the weight into a fixed-point format, determine, based on statistics of an activation of one or more layers configuring the neural network included in the parameter, a dynamic range of the activation, and quantize, based on statistics of input data of the neural network, the input data into a fixed-point format;
• and
• a neural processing unit (NPU) configured to perform inference by receiving the quantized weight, the dynamic range of the activation, and the quantized input data from the neural network quantization apparatus.

A processor-implemented method comprising:

• acquiring a parameter of a floating-point neural network model;
• calculating an Lp norm of a weight included in the parameter;
• deriving a relational expression between an Lp norm of an output activation and an Lp norm of an input activation of the floating-point neural network model based on the Lp norm of the weight;
• calculating a predicted value of the Lp norm of the output activation based on the Lp norm of the input activation;
• determining, based on the predicted value, a dynamic range of an activation of the floating-point neural network model;
• and
• quantizing the floating-point neural network model into a fixed-point neural network model based on the dynamic range.