Mixup data augmentation for knowledge distillation framework

A processing system for training a student neural network, comprising:

• one or more processor devices;
• a memory in communication with at least one of the one or more processor devices;
• and
• a display screen;
• wherein the memory includes:
• a teacher neural network configured to receive a data set including a plurality of input vectors and generate a plurality of output values;
• a softmax function configured to convert two of the plurality of output values from the teacher neural network for two corresponding input vectors into two corresponding soft labels;
• a mixup generator configured to combine the two corresponding input vectors to form a synthesized data vector;
• a masker configured to form a masked soft label vector from the two corresponding soft labels;
• and
• the student neural network configured to receive the synthesized data vector and masked soft label and train the student neural network based on an error calculated from the masked soft label.

A non-transitory computer readable storage medium comprising a computer readable program for training a student neural network, wherein the computer readable program when executed on a computer causes the computer to perform:

• feeding a data set including a plurality of input vectors into a teacher neural network to generate a plurality of output values;
• converting two of the plurality of output values from the teacher neural network for two corresponding input vectors into two corresponding soft labels;
• combining the two corresponding input vectors to form a synthesized data vector;
• forming a masked soft label vector from the two corresponding soft labels;
• feeding the synthesized data vector into the student neural network;
• using the masked soft label vector to determine an error for modifying weights of the student neural network;
• and
• modifying the weights of the student neural network.