Methods, mediums, and systems for training a model
AssigneeMeta Platforms, Inc.
InventorsPrince Gill, Honglei Liu, Wenhai Yang, Kshitiz Malik, Nanshu Wang, David Reiss
Life of this patent
Estimating the term…
Abstract
Exemplary embodiments relate to methods, mediums, and systems for moving language models from a server to the client device. Such embodiments may be deployed in an environment where the server is not able to provide modeling services to the clients, such as an end-to-end encrypted (E2EE) environment. Several different techniques are described to address issues of size and complexity reduction, model architecture optimization, model training, battery power reduction, and latency reduction.
Claims20
A method comprising:
- accessing a local model on an end-user device and a machine-learning-trained teacher model remote from the end-user device, the machine-learning-trained teacher model configured to teach a first task to the local model, the first task to comprise detecting when to trigger a suggestion, a recommendation, a reminder, or calling an agent, the local model being smaller in size than the teacher model;
- receiving a baseline output from the teacher model;
- and
- training the local model so that an output of the local model approximates the baseline output of the teacher model;
- wherein the local model is configured to be applied in an end-to-end encrypted environment.
A non-transitory computer-readable medium storing instructions configured to be executed by a processor to cause the processor to:
- access a local model on an end-user device and a machine-learning-trained teacher model remote from the end-user device, the machine-learning-trained teacher model configured to teach a first task to the local model, the first task to comprise detecting when to trigger a suggestion, a recommendation, a reminder, or calling an agent, the local model being smaller in size than the teacher model;
- receive a baseline output from the teacher model;
- and
- train the local model so that an output of the local model approximates the baseline output of the teacher model;
- wherein the local model is configured to be applied in an end-to-end encrypted environment.
An apparatus comprising:
- a network interface configured to:
- access a local model on an end-user device and a machine-learning-trained teacher model remote from the end-user device, the machine-learning-trained teacher model configured to teach a first task to the local model, the first task to comprise detecting when to trigger a suggestion, a recommendation, a reminder, or calling an agent, the local model being smaller in size than the teacher model, and wherein the local model is configured to be applied in an end-to-end encrypted environment, and
- receive a baseline output from the teacher model;
- a non-transitory computer-readable medium storing instructions configured to train the local model so that an output of the local model approximates the baseline output of the teacher model;
- and
- a processor configured to execute the instructions to provide a trainer.
Dependencies are inferred from the claim text.