Methods and systems for transferring knowledge from large language models to small language models using out-of-distribution feedbacks

A computer-implemented method for transferring knowledge from a Large Language Model (LLM) to a pre-trained Small Language Model (SLM) for training the SLM to execute a Natural Language Processing (NLP) task, the method comprising:

• during a first training iteration of a plurality of training iterations:
• acquiring, from a user, a client input including:
• (i) the NLP task for the pre-trained SLM, (ii) a plurality of data samples responsive to the NLP task, and (iii) the pre-trained SLM;
• training the pre-trained SLM to execute the NLP task based on the plurality of data samples, thereby generating a first trained SLM;
• determining, for each data sample of the plurality of data samples, a current prediction accuracy value of the SLM;
• identifying, in the plurality of data samples, at least one target data sample based on the current prediction accuracy value of the pre-trained SLM for each one of the plurality of data samples;
• generating an augmented plurality of data samples including the plurality of data samples and the at least one target data sample;
• during a second training iteration of the plurality of training iterations, following the first training iteration:
• generating, using the LLM, a second plurality of training samples for the first trained SLM based on the NLP task and the augmented plurality of data samples, in lieu of the plurality of data samples;
• training the first trained SLM based on the second plurality of training samples, thereby generating a second trained SLM;
• and
• causing a trained SLM, generated after the plurality of training iterations, to execute the NLP task in lieu of the LLM.

An electronic device for transferring knowledge from a Large Language Model (LLM) to a Small Language Model (SLM) for training the SLM to execute a Natural Language Processing (NLP) task, the electronic device comprising at least one processor and at least one non-transitory computer-readable memory storing executable instructions, which, when executed by the at least one processor, cause the electronic device to:

• during a first training iteration of a plurality of training iterations:
• acquire, from a user, a client input including:
• (i) the NLP task for the SLM, (ii) a plurality of data samples responsive to the NLP task, and (iii) an indication of the SLM;
• generate, using the LLM, a plurality of training samples for the SLM based on the NLP task and the plurality of data samples, the plurality of training samples being in-distribution training samples;
• train the SLM based on the plurality of training samples to execute the NLP task, thereby generating a first trained SLM;
• generate, using the LLM, a plurality of validation samples for the first trained SLM, the plurality of validation samples being out-of-distribution training samples with respect to the generated training sample in the first training iteration;
• determine, for each validation sample of the plurality of validation samples, a current prediction accuracy value of the SLM;
• select at least one target validation sample from the plurality of validation samples based on the current prediction accuracy value of the SLM for each of the plurality of validation samples;
• generate an augmented plurality of data samples including the plurality of data samples and the at least one target validation sample;
• and
• during a second training iteration of the plurality of training iterations, following the first training iteration:
• generate, using the LLM, a second plurality of training samples for the first trained SLM based on the NLP task and the augmented plurality of data samples, in lieu of the plurality of data samples;
• train the first trained SLM based on the second plurality of training samples, thereby generating a second trained SLM;
• and
• cause an updated SLM, generated after the plurality of training iterations, to execute the NLP task in lieu of the LLM.