Task processing method and apparatus based on model quantization, and device and storage medium

An electronic device, comprising a processor and a memory, wherein the memory stores machine-executable instructions executable by the processor, and the processor is configured to execute the machine-executable instructions to:

• for any one of optimization units in a Transformer model,
• obtain a target weight quantization factor and a target activation quantization factor of the optimization unit by updating, with a goal of minimizing a first difference between a first quantized output and a first floating-point output of the optimization unit, a weight quantization factor and an activation quantization factor of the optimization unit, wherein
• the first quantized output comprises an output result of the optimization unit when a weight parameter of the optimization unit is quantized based on the weight quantization factor and an input and an output of the optimization unit are quantized based on the activation quantization factor, and
• the first floating-point output comprises an output result of the optimization unit when the weight parameter of the optimization unit is a floating-point weight parameter and the input is a floating-point input;
• in a case that the floating-point weight parameter, the target weight quantization factor and the target activation quantization factor of the optimization unit are fixed, obtain a target weight quantization increment of the optimization unit by updating, with a goal of minimizing a second difference between a second quantized output and a second floating-point output of the optimization unit, a weight quantization increment of the optimization unit, wherein
• the second quantized output comprises an output result of the optimization unit when the weight parameter of the optimization unit is quantized based on the target weight quantization factor and quantization adjustment is performed based on the weight quantization increment, and the input and the output of the optimization unit are quantized based on the target activation quantization factor, and
• the second floating-point output comprises an output result of the optimization unit when the weight parameter of the optimization unit is the floating-point weight parameter and the input is a floating-point input;
• determine a weight quantization rounding direction of the optimization unit based on the target weight quantization increment;
• obtain a target quantization weight parameter of the optimization unit by quantizing the weight parameter of the optimization unit based on the target weight quantization factor and the weight quantization rounding direction;
• and
• in a case that the Transformer model is used for task processing, for any one of the optimization units,
• perform forward inference computation on input data of the optimization unit based on the target quantization weight parameter for the optimization unit, and
• quantize the input and the output of the optimization unit based on the target activation quantization factor of the optimization unit.