# US 2025/0095641 A1 · Large language model (llm)-based correction based on a multi-tool prompt > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2025/0095641 A1 - Document ID: US20250095641A1 - Status: Published application - Kind: A1 - Application number: 18/369,049 - Filed: 2023-09-15 - Published: 2025-03-20 - Earliest claimed: 2023-09-15 - Assignees: Amazon Technologies, Inc. - Applicants: Amazon Technologies, Inc. - Inventors: Deepak Babu Rajaram Piskala - Web page: https://patentgrep.com/patent/US20250095641A1 ## CPC - G06F 16/3329: Natural language query formulation - G10L 15/183: using context dependencies, e.g. language models - G06F 16/3344: {using natural language analysis} - G06F 16/338: Presentation of query results - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract Techniques for large language model (LLM)-based correction based on a multi-tool prompt are described. In an example, a computer system receives, via a user interface, user input including user-provided information and indicating a request for a task to be performed on the user-provided information. The computer system generates, by using an LLM associated with a prompt, a first input to a first tool based on the user input. The prompt indicating a sequence of steps to perform for the task and tools available to the LLM. The first tool corresponds to a first step of the sequence of steps. The computer system determines, by using the LLM, a first output of the first tool in response to the first input and an update to the user-provided information based on the first output and a completion of the task. The computer system causes the user interface to present the update. ## Claims ### Claim 1 A computer-implemented method comprising: receiving, via a user interface, user input including user-provided information and indicating a request for a task to be performed on the user-provided information, the task comprising text correction, the user-provided information comprising text that is output by an automatic speech recognition (ASR) process and that includes a first entity identifier; generating, by using a large language model associated with a prompt, a first input to a first tool based at least in part on the user input, the prompt indicating a sequence of steps to perform for the task, a plurality of tools available to the large language model, and an execution format for each step, the first tool corresponding to a first step of the sequence of steps, the first input being based at least in part on a first execution format for the first step; determining, by using the large language model, a first output of the first tool in response to the first input; determining, by using the large language model, that a second step of the sequence is to be performed based at least in part on the first output; generating, by using the large language model, a second input to a second tool based at least in part on the first output, the second tool corresponding to the second step, the second input being based at least in part on a second execution format for the second step; determining, by using the large language model, a second output of the second tool in response to the second input; determining, by using the large language model, an update to the user-provided information based at least in part on the second output and a completion of the task, the update comprising a correction to the first entity identifier; and causing the user interface to present the update. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, further comprising: sending, by using the large language model, an application programming interface (API) call via an API to the first tool, wherein the API call includes the first input, wherein the first tool comprises a machine learning model, and wherein the first output is determined based at least in part on an API response of the machine learning model; and sending, by using the large language model, a search request via a web interface to the second tool, wherein the search request includes the second input, wherein the second tool comprises a search engine, and wherein the second output is determined based at least in part on a search result of the search engine. ### Claim 3 (depends on 1) The computer-implemented method of claim 1, further comprising: determining, by using the large language model, that performing the task is incomplete based at least in part on the second output; generating, by using the large language model, a third input to a third tool based at least in part on the second output, the third tool corresponding to a third step, the third input being based at least in part on a third execution format for the third step; determining, by using the large language model, a third output of the third tool in response to the third input, the third output corresponding to different user input via a different user interface; and determining, by using the large language model, the completion of the task based at least in part on the third output, wherein the update is generated based at least in part on the third output. ### Claim 4 (depends on 1) The computer-implemented method of claim 1, further comprising: providing, by using the large language model, a third input to a different large language model, wherein the third input includes the user input, the first input, the first output, the second input, the second output, and the update; and receiving reinforcement information from the different large language model based at least in part on the third input. ### Claim 5 A system comprising: one or more processors; and one or more memory storing instructions that, upon execution by the one or more processors, configure the system to: receive, via a user interface, user input including user-provided information and indicating a request for a task to be performed on the user-provided information; generate, by using a large language model associated with a prompt, a first input to a first tool based at least in part on the user input, the prompt indicating a sequence of steps to perform for the task and a plurality of tools available to the large language model, the first tool corresponding to a first step of the sequence of steps; determine, by using the large language model, a first output of the first tool in response to the first input; determine, by using the large language model, an update to the user-provided information based at least in part on the first output and a completion of the task; and cause the user interface to present the update. ### Claim 6 (depends on 5) The system of claim 5, wherein the task comprises text verification, wherein the user-provided information comprises text that is output by an automatic speech recognition (ASR) process and that includes a first entity identifier, and wherein the update comprises a correction to the first entity identifier. ### Claim 7 (depends on 5) The system of claim 5, wherein the prompt further indicates an execution format for the first step. ### Claim 8 (depends on 7) The system of claim 7, wherein the execution format indicates procedure that includes a question to be answered, a thought about a next step, an action to take based at least in part on the plurality of tools, an action input to generate, and an observation to generate as a result of the action. ### Claim 9 (depends on 8) The system of claim 8, wherein the execution format indicates a number of times the procedure can be repeated, and a final answer to generate and present upon determination of a correctness of the final answer. ### Claim 10 (depends on 5) The system of claim 5, wherein the one or more memory store further instructions that, upon execution by the one or more processors, further configure the system to: determine, by using the large language model, that a second step of the sequence is to be performed based at least in part on the first output; generate, by using the large language model, a second input to a second tool based at least in part on the first output, the second tool corresponding to the second step, the second input being based at least in part on a second execution format for the second step; and determine, by using the large language model, a second output of the second tool in response to the second input, wherein the update is further based at least in part on the second output. ### Claim 11 (depends on 10) The system of claim 10, wherein the one or more memory store additional instructions that, upon execution by the one or more processors, further configure the system to: determine, by using the large language model, that performing the task is incomplete based at least in part on the first output, wherein a determination that the second step is to be performed is based at least in part on the performing being incomplete. ### Claim 12 (depends on 5) The system of claim 5, wherein the one or more memory store further instructions that, upon execution by the one or more processors, further configure the system to: determine, by using the large language model, that performing the task is complete based at least in part on the first output; and forego execution of a next step indicated by the prompt based at least in part on the performing being complete. ### Claim 13 One or more computer-readable storage media storing instructions that, upon execution on a system, cause the system to perform operations comprising: receiving, via a user interface, user input including user-provided information and indicating a request for a task to be performed on the user-provided information; generating, by using a large language model associated with a prompt, a first input to a first tool based at least in part on the user input, the prompt indicating a sequence of steps to perform for the task and a plurality of tools available to the large language model, the first tool corresponding to a first step of the sequence of steps; determining, by using the large language model, a first output of the first tool in response to the first input; determining, by using the large language model, an update to the user-provided information based at least in part on the first output and a completion of the task; and causing the user interface to present the update. ### Claim 14 (depends on 13) The one or more computer-readable storage media of claim 13, wherein the operations further comprise: storing task-related data in a memory accessible to the large language model, wherein the task-related data comprises the user input, the first input, the first output, and the update, wherein re-performing the task is based at least in part on the task-related data stored in the memory. ### Claim 15 (depends on 13) The one or more computer-readable storage media of claim 13, wherein the user input and the request are a first user input and a first request, respectively, and wherein the operations further comprise: storing task-related data in a memory cache; receiving, a second user input indicating a second request for the task to be performed again; and determining whether a response to the second user input is available from the memory cache. ### Claim 16 (depends on 15) The one or more computer-readable storage media of claim 15, wherein the operations further comprise: causing, based at least in part on the response being unavailable from the memory cache, the large language model to perform the task based at least in part on the second user input. ### Claim 17 (depends on 15) The one or more computer-readable storage media of claim 15, wherein the operations further comprise: generating, based at least in part on the response being available from the memory cache, the response from the task-related data, wherein the response includes the update; and foregoing performing the task again in response to the second user input. ### Claim 18 (depends on 13) The one or more computer-readable storage media of claim 13, wherein the operations further comprise: storing task-related data in a memory accessible to the large language model; providing, by using the large language model, a second input to a different large language model, wherein the second input includes the task-related data; and receiving reinforcement information from the different large language model based at least in part on the second input. ### Claim 19 (depends on 13) The one or more computer-readable storage media of claim 13, wherein the operations further comprise: generating, by using the large language model, a second input to a second tool based at least in part on the first output, the second tool corresponding to a second step of the sequence of steps, the second input being based at least in part on a second execution format for the second step; and determining, by using the large language model, a second output of the second tool in response to the second input, wherein the update is further based at least in part on the second output. ### Claim 20 (depends on 19) The one or more computer-readable storage media of claim 19, wherein the operations further comprise: sending, by using the large language model, an application programming interface (API) call via an API to the first tool, wherein the API call includes the first input, and wherein the first output is determined based at least in part on an API response of the first tool; and sending, by using the large language model, an output request via a web interface to the second tool, wherein the output request includes the second input, and wherein the second output is determined based at least in part on a response of the second tool.