Back to results
US 2023/0409826 A1Published application
Training a computer-implemented conditional language model for improved performance
AssigneeMicrosoft Technology Licensing, LLC
InventorsJunyi CHAI, Konstantin GOLOBOKOV, Ye DONG, Reid Allen PRYZANT, Yi LIU
Life of this patent
Estimating the term…
Abstract
Technologies related to computer-implemented conditional language models (CLMs) are described. A first CLM is trained to generate output texts based upon input texts and conditions. Output texts generated by the first CLM are included in a training set, and a second CLM is trained based upon the training set. The second CLM is then configured to receive input text and a condition and generate an output text based upon the input text and the condition.
Claims20
A computing system that is configured to train a language model, the computing system comprising:
- a processor;
- and
- memory storing instructions that, when executed by the processor, cause the processor to perform acts comprising:
- providing, as input to a first computer-implemented language model:
- text;
- and
- a condition, wherein the first computer-implemented language model generates output text based upon the text and the condition, and further wherein the condition corresponds to a desired attribute value for the output text;
- providing the output text as input to a computer-implemented classifier, wherein the computer-implemented classifier generates an output based upon the output text, and further wherein the output is indicative of an actual attribute value for the output text;
- based upon the output of the classifier, determining that the actual attribute value of the output text identified by the classifier is equivalent to the desired attribute value of the output text;
- upon determining that the actual attribute value is equivalent to the desired attribute value, including the output text in training data, wherein the output text is labeled with the actual attribute value;
- and
- training a second computer-implemented language model based upon the training data, wherein the second computer-implemented model, when trained, is configured to receive texts and corresponding conditions as input and generate output texts based upon the texts and the corresponding conditions.
9 more elements
A method executed by a computer processor, the method comprising:
- providing text and a condition to a first computer-implemented conditional language model (CLM), wherein the first CLM generates output text having a value for an attribute based upon the text and the condition, and further wherein the condition specifies a desired value for the attribute;
- providing the output text generated by the first CLM to a classifier, wherein the classifier identifies, based upon the output text, the value for the attribute of the output text from amongst several potential values for the attribute;
- performing a comparison between the value for the attribute identified by the classifier with the desired value for the attribute specified by the condition;
- based upon the comparison, determining that the value for the attribute identified by the classifier matches the desired value for the attribute specified by the condition;
- upon determining that the value for the attribute identified by the classifier matches the value for the attribute specified by the condition, including the output text and the value for the attribute identified by the classifier in training data;
- and
- training a second CLM model based upon the training data, wherein the second CLM model, when trained, is configured to receive texts and corresponding conditions as input and generate output texts based upon the texts and corresponding conditions.
4 more elements
A computer-readable storage medium comprising instructions that, when executed by a processor, cause the processor to perform acts comprising:
- providing, as input to a first computer-implemented language model:
- text;
- and
- a condition, wherein the first computer-implemented language model generates output text based upon the text and the condition, and further wherein the condition corresponds to a desired attribute value for the output text;
- providing the output text as input to a computer-implemented classifier, wherein the computer-implemented classifier generates an output based upon the output text, and further wherein the output is indicative of an actual attribute value for the output text;
- based upon the output of the classifier, determining that the actual attribute value of the output text identified by the classifier is equivalent to the desired attribute value of the output text;
- upon determining that the actual attribute value is equivalent to the desired attribute value, including the output text in training data, wherein the output text is labeled with the actual attribute value;
- and
- training a second computer-implemented language model based upon the training data, wherein the second computer-implemented model, when trained, is configured to receive texts and corresponding conditions as input and generate output texts based upon the texts and the corresponding conditions.
6 more elements
Dependencies are inferred from the claim text.