Distilling language models
AssigneeZoom Communications, Inc.
InventorsShamil Chollampatt Muhammed Ashraf, Sathish Reddy Indurthi, Minh-Quang Pham, Marco Turchi
Life of this patent
Estimating the term…
Abstract
One example method includes selecting an entry from a validation set based on similarity to a first communication record, wherein the entry from the validation set includes a second communication record and a corresponding summary of the second communication record; generating a prompt that includes the first communication record, the second communication record, and the corresponding summary of the second communication record; inputting the prompt to a first language model to obtain a generated summary of the first communication record; and training a second language model based on the first communication record and the generated summary.
Claims20
A method comprising:
- fine-tuning a small language model, comprising:
- accessing a set of communication records and a set of validation records;
- for each communication record in the set of communication records, generating a first embedding representing the respective communication record;
- for each validation record in the set of communication records, generating a second embedding representing the respective validation record;
- for each communication record:
- determining a similarity between the respective first embedding and each generated second embedding;
- selecting a validation record from the set of validation records based on the determined similarities;
- and
- generating, using a large language model (“LLM”), a summary of the respective communication record comprising inputting the respective communication record and the validation record into the LLM;
- and
- training the small language model comprising prompting the small language model to generate a summary of a first communication record and providing the first communication record and the generated summary as a prompt/completion pair.
A system comprising:
- a non-transitory computer-readable medium;
- and
- one or more processors communicatively coupled to the non-transitory computer-readable medium, the one or more processors configured to execute processor-executable instructions stored in the non-transitory computer-readable medium to:
- fine-tune a small language model, the one or more processors configured to execute processor-executable instructions configured to cause one or more processors to:
- access a set of communication records and a set of validation records;
- for each communication record in the set of communication records, generate a first embedding representing the respective communication record;
- for each validation record in the set of communication records, generate a second embedding representing the respective validation record;
- for each communication record:
- determine a similarity between the respective first embedding and each generated second embedding;
- select a validation record from the set of validation records based on the determined similarities;
- and
- generate, using a large language model (“LLM”), a summary of the respective communication record comprising inputting the respective communication record and the validation record into the LLM;
- and
- train the small language model comprising prompting the small language model to generate a summary of a first communication record and providing the first communication record and the generated summary as a prompt/completion pair.
A non-transitory computer-readable medium comprising processor-executable instructions configured to cause one or more processors to:
- fine-tune a small language model, comprising processor-executable instructions configured to cause one or more processors to:
- access a set of communication records and a set of validation records;
- for each communication record in the set of communication records, generate a first embedding representing the respective communication record;
- for each validation record in the set of communication records, generate a second embedding representing the respective validation record;
- for each communication record:
- determine a similarity between the respective first embedding and each generated second embedding;
- select a validation record from the set of validation records based on the determined similarities;
- and
- generate, using a large language model (“LLM”), a summary of the respective communication record comprising inputting the respective communication record and the validation record into the LLM;
- and
- train the small language model comprising prompting the small language model to generate a summary of a first communication record and providing the first communication record and the generated summary as a prompt/completion pair.
Dependencies are inferred from the claim text.