Synthetic data generation using large language models
AssigneeNVIDIA Corporation
InventorsChandana NEERUKONDA, Rajath SHETTY, Felipe SOARES, Makesh Narsimhan SREEDHAR
Life of this patent
Estimating the term…
Loading patent…
AssigneeNVIDIA Corporation
InventorsChandana NEERUKONDA, Rajath SHETTY, Felipe SOARES, Makesh Narsimhan SREEDHAR
Estimating the term…
In various examples, synthetic question-answer (QA) pairs may be generated using question and answer generation models comprising corresponding language models (e.g., autoregressive LLMs). A repository of textual data representing a particular knowledge base may be used to source synthetic QA pairs by partitioning textual data from the repository into units of text (e.g., paragraphs) that represent context. For each unit of text, the question generation model may be prompted to generate a synthetic question from that unit of text, and the answer generation model may be prompted to generate a synthetic answer to the synthetic question. Textual entailment and/or human evaluations may be used to filter out low quality, incorrect, and/or non-productive QA pairs that may be a result of hallucinations. As such, the synthetic QA pairs may be used as, and/or may be used to generate, training data for one or more machine learning models.
A processor comprising:
A system comprising one or more processing units to generate a synthetic question based at least on applying a representation of one or more sequences of source text from a repository to one or more first language models, and generate a synthetic answer to the synthetic question based at least on applying a representation of the synthetic question to one or more second language models.
A method comprising:
Dependencies are inferred from the claim text.