Methods, devices and media providing an integrated teacher-student system

A method, comprising:

• receiving a feature map;
• generating positional normalization data based on the feature map;
• down-sampling the feature map to generate an embedding vector;
• generating compressed data comprising the positional normalization data and the embedding vector;
• and
• transmitting the compressed data over a communication link.

A method, comprising:

• providing an integrated system comprising:
• a teacher neural network comprising, in series:
• a first teacher sub-network adapted to generate a first intermediate feature map based on a training-mode data input to the teacher neural network;
• and
• a final teacher sub-network adapted to generate teacher inference data based on the first intermediate feature map;
• and
• a first student sub-network adapted to generate first student inference data based on the first intermediate feature map;
• and
• training the integrated system by:
• propagating the training-mode data input forward through the teacher neural network to generate the first intermediate feature map and teacher inference data;
• propagating the first intermediate feature map forward through the first student sub-network to generate first student inference data;
• calculating a first knowledge distillation loss based on a knowledge distillation loss function applied to the first student inference data and the teacher inference data;
• and
• propagating the first knowledge distillation loss backward through the first student sub-network and the first teacher sub-network to train the first student sub-network and the first teacher sub-network.