Adaptive Large Language Model Selection And Refinement For Extracting Data From Documents
AssigneeOracle International Corporation
InventorsChristopher S. Thompson, Daniel John Driscoll, Samuel Aaron Witt
Life of this patent
Estimating the term…
Loading patent…
AssigneeOracle International Corporation
InventorsChristopher S. Thompson, Daniel John Driscoll, Samuel Aaron Witt
Estimating the term…
Techniques are described herein for adaptively selecting and deploying language models, such as large language models (LLMs), to extract data from electronic documents. User overrides of extracted data are tracked and used to compute accuracy benchmarks for multiple language models. The benchmark data may drive the selection of which language model is used to extract data in a given context. The process may select different language models for different contexts depending on which language model is most accurate for the given context. Attributes other than accuracy, such as cost and latency, may also be a factor in which language model is selected. The adaptive approach allows for ongoing improvement in data extraction through reinforcement feedback while optimizing for one or more target factors, such as model accuracy, latency, and/or cost.
A method comprising:
One or more non-transitory computer readable media storing instructions that, when executed by one or more hardware processors, cause performance of a set of operations comprising:
Dependencies are inferred from the claim text.