Separate quantization method of forming combination of 4-bit and 8-bit data of neural network

A separate quantization method of forming a combination of 4-bit and 8-bit data of a neural network, comprising,

• inputting a trained neural network model, and determining whether at least one training data set and at least one validation data set exist;
• when the at least one training data set or the at least one validation data set exists, using a calibration manner to determine a threshold for activations of each of a plurality of layers of the neural network model, so as to determine how many of the activations to perform 8-bit quantization and determine the remaining activations to perform 4-bit quantization;
• when the at least one training data set and the at least one validation data set do not exist, uniformly allocating the activations of each of the plurality of layers of the neural network model to perform 4-bit quantization and 8-bit quantization;
• quantizing weights of the neural network model, and when the at least one training data set and the at least one validation data set exist, allocating the weights of each of the plurality of layers to 4-bit weights and 8-bit weights according to a predetermined ratio, and when any one of the training data set and the validation data set does not exist, uniformly allocating the weights of each of the plurality of layers to the 4-bit weights and the 8-bit weights;
• and
• obtaining an optimized neural network model with 4-bit and 8-bit quantized weights.