Neural network model compression method and apparatus, storage medium, and chip

A neural network model compression apparatus, comprising a memory and a processor, wherein the memory is configured to store a computer program, and the processor is configured to invoke the computer program from the memory and run the computer program to perform a neural network compression method comprising:

• obtaining, by a server, a first neural network model and training data of the first neural network model that are uploaded by user equipment;
• obtaining, by the server, a positive-unlabeled (PU) classifier by using a PU learning algorithm based on the training data of the first neural network model and unlabeled data stored in the server;
• selecting, by the server by using the PU classifier, extended data from the unlabeled data stored in the server, wherein the extended data is positive sample data determined by the PU classifier and has a property and distribution similar to a property and distribution of the training data of the first neural network model;
• training, by the server, a second neural network model by using a knowledge distillation (KD) method based on the extended data, wherein the first neural network model is used as a teacher network model of the KD method and the second neural network model is used as a student network model of the KD method;
• and
• wherein the PU classifier is obtained based on a first feature and a proportion information, the first feature is obtained based on fusion of a plurality of third features and the plurality of third features are in a one-to-one correspondence with a plurality of layers of the first neural network;
• and wherein the processor is further configured to:
• input, by the server, the extended data into the first neural network model, to classify the extended data and obtain extended data of the plurality of classes and a second weight of extended data of each of the plurality of classes;
• and
• minimize, by the server, a loss function of the KD method, to obtain a trained second neural network model, wherein the loss function of the KD method is a sum of products of training errors of extended data of all of the plurality of classes and second weights of the extended data of all the classes;
• wherein the second weights of the extended data of all the classes comprise a plurality of perturbed weights obtained after random perturbation is performed on initial weights of the extended data of all the classes, and the loss function of the KD method comprises a plurality of loss functions in a one-to-one correspondence with the plurality of perturbed weights, wherein an initial weight of the extended data of each class is in negative correlation with an amount of the extended data of each class;
• and the processor is further configured to:
• minimize, by the server, maximum values of the plurality of loss functions, to obtain the trained second neural network model.

A non-transitory computer-readable storage medium, wherein the non-transitory computer-readable medium stores program code executed by a device, and the program code is used to perform the method, comprising:

• obtaining, by a server, a first neural network model and training data of the first neural network model that are uploaded by user equipment;
• obtaining, by the server, a positive-unlabeled (PU) classifier by using a PU learning algorithm based on the training data of the first neural network model and unlabeled data stored in the server;
• selecting, by the server by using the PU classifier, extended data from the unlabeled data stored in the server, wherein the extended data is positive sample data determined by the PU classifier and has a property and distribution similar to a property and distribution of the training data of the first neural network model;
• training, by the server, a second neural network model by using a knowledge distillation (KD) method based on the extended data, wherein the first neural network model is used as a teacher network model of the KD method and the second neural network model is used as a student network model of the KD method;
• and
• wherein the PU classifier is obtained based on a first feature and a proportion information, the first feature is obtained based on fusion of a plurality of third features and the plurality of third features are in a one-to-one correspondence with a plurality of layers of the first neural network;
• and the method further comprising:
• inputting, by the server, the extended data into the first neural network model, to classify the extended data and obtain extended data of the plurality of classes and a second weight of extended data of each of the plurality of classes;
• and
• minimizing, by the server, a loss function of the KD method, to obtain a trained second neural network model, wherein the loss function of the KD method is a sum of products of training errors of extended data of all of the plurality of classes and second weights of the extended data of all the classes;
• and
• wherein the second weights of the extended data of all the classes comprise a plurality of perturbed weights obtained after random perturbation is performed on initial weights of the extended data of all the classes, and the loss function of the KD method comprises a plurality of loss functions in a one-to-one correspondence with the plurality of perturbed weights, wherein an initial weight of the extended data of each class is in negative correlation with an amount of the extended data of each class;
• and
• minimizing, by the server, maximum values of the plurality of loss functions, to obtain the trained second neural network model.