US 2026/0220029 A1Published application

Agentic test-driven code generation for insight discovery

AssigneeOracle International Corporation

InventorsMoein Owhadi Kareshk, Achraf Zhar, Yasha Pushak, Hesam Fathi Moghadam, Rhicheek Patra, Hassan Chafi

Life of this patent

Estimating the term…

Loading patent…

Abstract

A computer generates and iteratively improves operational logic by incremental refinement of a linguistic prompt to increase its accuracy or by full replacement of the linguistic prompt by a somewhat unrelated linguistic prompt. From a natural language (NL) question about data, a problem summarization large language model (LLM) generates an NL problem summary that specifies multiple semantic requirements. From the semantic requirements, a prover LLM generates a test case that can detect an error in an answer for the NL question. From the semantic requirements, a strategy LLM generates multiple NL solution summaries that each describes a respective distinct solution. An initially best solution summary is iteratively reused to generate and refine solution logic. Excessive iterating may cause switching to a different solution summary for solution logic generation. The generated test cases ensure that logic generation continues until an answer for the NL question is obtained that is provably correct.

Claims20

Dependent claims

A method comprising:

inferentially generating, from a natural language (NL) question about a data, by a problem summarization large language model (LLM), an NL problem summary that specifies a plurality of semantic requirements;
inferentially generating, from the plurality of semantic requirements, by a prover LLM, a test case that can detect an error in an answer for the NL question;
inferentially generating, from the plurality of semantic requirements, by a strategy LLM, a plurality of NL solution summaries, wherein each NL solution summary describes a respective distinct solution of a plurality of mutually-exclusive solutions for the NL question;
inferentially generating, from a first selected NL solution summary of the plurality of NL solution summaries, by a first scripting LLM, an incorrect logic that cannot generate a correct answer;
detecting an error of the incorrect logic, wherein said detecting can comprise executing the incorrect logic and executing the test case;
selecting, based on said error, a second selected NL solution summary of the plurality of NL solution summaries, wherein the second selected NL solution summary can be the first selected NL solution summary;
generating, based on the second selected NL solution summary of the plurality of NL solution summaries, a new linguistic prompt that specifies the plurality of semantic requirements;
inferentially generating, from the new linguistic prompt, by a second scripting LLM that can be the first scripting LLM, a correct logic that can read said data and generate a correct answer;
and
executing the correct logic, including reading said data, including generating the correct answer.

7 more elements

One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause:

inferentially generating, from a natural language (NL) question about a data, by a problem summarization large language model (LLM), an NL problem summary that specifies a plurality of semantic requirements;
inferentially generating, from the plurality of semantic requirements, by a prover LLM, a test case that can detect an error in an answer for the NL question;
inferentially generating, from the plurality of semantic requirements, by a strategy LLM, a plurality of NL solution summaries, wherein each NL solution summary describes a respective distinct solution of a plurality of mutually-exclusive solutions for the NL question;
inferentially generating, from a first selected NL solution summary of the plurality of NL solution summaries, by a first scripting LLM, an incorrect logic that cannot generate a correct answer;
detecting an error of the incorrect logic, wherein said detecting can comprise executing the incorrect logic and executing the test case;
selecting, based on said error, a second selected NL solution summary of the plurality of NL solution summaries, wherein the second selected NL solution summary can be the first selected NL solution summary;
generating, based on the second selected NL solution summary of the plurality of NL solution summaries, a new linguistic prompt that specifies the plurality of semantic requirements;
inferentially generating, from the new linguistic prompt, by a second scripting LLM that can be the first scripting LLM, a correct logic that can read said data and generate a correct answer;
and
executing the correct logic, including reading said data, including generating the correct answer.

7 more elements

Dependencies are inferred from the claim text.