Systems and methods for language model-based text insertion
AssigneeOpenAI OpCo, LLC
InventorsMohammad Bavarian, Heewoo Jun
Life of this patent
Estimating the term…
Abstract
Disclosed herein are methods, systems, and computer-readable media for automatically generating and inserting text. In an embodiment, a method may include receiving an input text prompt comprising a prefix portion and a suffix portion. The method may also include accessing a language model based on the input text prompt, and determining a set of context parameters based on the input text prompt and the language model. The method may also include generating an output text prompt based on the set of context parameters and the language model, and inserting the output text prompt into the input text prompt.
Claims20
A system comprising:
- at least one memory storing instructions;
- at least one processor configured to execute the instructions to perform operations for automatically generating and inserting text, the operations comprising:
- receiving an input text prompt comprising a prefix portion and a suffix portion;
- determining a set of model parameters based on the input text prompt;
- accessing a language model based on the input text prompt and the set of model parameters;
- determining a set of context parameters based on the input text prompt and the language model, the set of context parameters comprising at least one of location, person, time, or event;
- generating language model output text based on the set of context parameters and the language model;
- inserting the language model output text into the input text prompt based on the language model;
- and
- optimizing the accessed language model by aligning the language model based on the language model output using machine learning,
- wherein:
- the language model is optimized through one or more iterative cycles of training based on one or more outcome metrics associated with the language model output and one or more datasets, and
- the one or more datasets comprise at least one of user-instruction data or user-labeled data based on language model output text.
A method for automatically generating and inserting text, comprising:
- receiving an input text prompt comprising a prefix portion and a suffix portion;
- determining a set of model parameters based on the input text prompt;
- accessing a language model based on the input text prompt and the set of model parameters;
- determining a set of context parameters based on the input text prompt and the language model, the set of context parameters comprising at least one of location, person, time, or event;
- generating language model output text based on the set of context parameters and the language model;
- inserting the language model output text into the input text prompt based on the language mode;
- and
- optimizing the accessed language model by aligning the language model based on the language model output using machine learning;
- wherein:
- the language model is optimized through one or more iterative cycles of training based on one or more outcome metrics associated with the language model output and one or more datasets, and
- the one or more datasets comprise at least one of user-instruction data or user-labeled data based on language model output text.
Dependencies are inferred from the claim text.