Back to results
US 12,614,030 B2Granted
Enriching language model input with contextual data
AssigneeMicrosoft Technology Licensing, LLC
InventorsHaim Somech, Adi L. Miller, Assaf Avihoo
Life of this patent
Estimating the term…
Abstract
Various embodiments discussed herein are directed to improving existing technologies by providing a corpus data supplement as input into a model, such as a Large Language Model (LLM). Consequently, the model can generate accurate scores or data for predictions because the model is better able to distinguish between a general understanding of natural language concepts and domain-specific concepts.
Claims20
A system comprising:
- at least one computer processor;
- and
- one or more computer storage media storing computer-useable instructions that, when used by the at least one computer processor, cause the at least one computer processor to perform operations comprising:
- receiving a corpus of text;
- determining contextual data associated with the corpus of text;
- based on the contextual data, determining a corpus data supplement, the corpus data supplement being at least one of:
- data to be added within the corpus of text as input into a trained machine learning model or data that supplements the corpus of text as input into the trained machine learning model, wherein determining the corpus data supplement comprises determining a first portion, among other portions, of the contextual data to be included in the corpus data supplement by determining contextual data from more relevant to less relevant until a token threshold is met, the token threshold corresponding to an input size constraint of the trained machine learning model, wherein the other portions are excluded from the corpus data supplement based on the token threshold having been met and being less relevant relative to the first portion;
- based on the determining of the corpus data supplement, providing the corpus of text and the corpus data supplement as input into the trained machine learning model;
- and
- based on the providing of the corpus of text and the corpus data supplement as input into the trained machine learning model, causing the machine learning model to perform an output that is based on the corpus data supplement.
7 more elements
A computer-implemented method comprising:
- receiving a corpus of text;
- determining contextual data associated with the corpus of text, wherein determining the corpus data supplement comprises:
- detecting a plurality of entities within the corpus of text by scanning the corpus of text;
- scoring each entity of the plurality of entities according to a relevance of the entity to the contextual data;
- ranking each entity of the plurality of entities based on the score;
- and
- for each entity of the plurality of entities:
- based on the ranking, determining an entity type;
- determining whether to enrich the entity based on the entity type and a size of at least one of:
- the corpus of text, a portion of the contextual data, or an entity data enrichment;
- and
- based on determining to enrich the entity, determining an entity data enrichment data based on the contextual data;
- based on the contextual data, determining a corpus data supplement;
- subsequent to the determining of the corpus data supplement, receiving a user query;
- in response to the receiving of the user query, providing the user query, the corpus of text, and the corpus data supplement as input into a machine learning model;
- and
- based on the providing, causing the user query to be executed such that one or more results for the query are returned, wherein the one or more results are based at least on the corpus data supplement.
14 more elements
One or more computer storage media having computer-executable instructions embodied thereon that, when executed, by one or more processors, cause the one or more processors to perform operations comprising:
- receiving a corpus of text;
- determining contextual data associated with the corpus of text;
- based on the contextual data, determining a corpus data supplement, the corpus data supplement being at least one of:
- data to be added within the corpus of text or data to be supplemented with the corpus of text as input into a language model, wherein determining the corpus data supplement comprises determining a first portion, among other portions, of the contextual data to be included in the corpus data supplement by determining contextual data from more relevant to less relevant until a token threshold is met, the token threshold corresponding to an input size constraint of the trained machine learning model, wherein the other portions are excluded from the corpus data supplement based on the token threshold having been met and being less relevant relative to the first portion;
- based on the determining of the corpus data supplement, causing the corpus of text and the corpus data supplement to be used as input into the language model;
- and
- receiving, from the language model, an output that is based on the corpus data supplement.
4 more elements
Dependencies are inferred from the claim text.