# US 2025/0094189 A1 · Digital assistant with copilot support to enhance application usage

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0094189 A1
- Document ID: US20250094189A1
- Status: Published application
- Kind: A1
- Application number: 18/885,492
- Filed: 2024-09-13
- Published: 2025-03-20
- Earliest claimed: 2023-09-15
- Assignees: Oracle International Corporation
- Applicants: Oracle International Corporation
- Inventors: Sunghye Jeon; Steven Martijn Davelaar
- Web page: https://patentgrep.com/patent/US20250094189A1

## CPC

- G16H 10/00: ICT specially adapted for the handling or processing of patient-related medical or healthcare data (for medical reports G16H15/00; for therapies or health-improving plans G16H20/00; for the handling or processing of medical images G16H30/00)
- G16H 10/60: for patient-specific data, e.g. for electronic patient records
- G06F 3/0482: Interaction with lists of selectable items, e.g. menus
- G06F 3/0483: Interaction with page-structured environments, e.g. book metaphor
- G06F 9/453: {Help systems}
- G06F 9/546: {Message passing systems or structures, e.g. queues}
- G06F 40/30: Semantic analysis
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Techniques for implementing a digital assistant with copilot support to enhance application usage. In one aspect, a method includes receiving a message payload, invoking, using a thread, a flow based on the message payload, generating a context variable data structure associated with the thread, responsive to invoking the flow, determining, using a machine learning model, an intent of the user, accessing, based on the intent, a prompt and an object schema, and revising the prompt based on the message payload, data in the context variable data structure, and the object schema. A generative artificial intelligence model then generates a list comprising one or more executable actions based on the prompt. The one or more executable actions are executed based on one or more parameters to obtain an output, and the output or a communication derived from the output are then sent to a user.

## Claims

### Claim 1

A computer-implemented method comprising:
receiving a message payload comprising: (i) an utterance or indication of an interface action, and (ii) application context, associated with an interaction between a user and an application;
invoking, using a thread, a flow based on the message payload;
generating a context variable data structure associated with the thread, wherein generating the context variable data structure comprises storing one or more variables from the application context as at least a portion of data stored within the context variable data structure;
responsive to invoking the flow, determining, using a machine learning model, an intent of the user based on (i) the utterance or the indication of the interface action, (ii) the data in the context variable data structure, or (iii) both;
accessing, based on the intent, a prompt and an object schema;
inserting: the utterance or the indication of the interface action, at least a portion of the data in the context variable data structure, and the object schema into the prompt;
generating, using a generative artificial intelligence (GenAI) model, a list comprising one or more executable actions based on the prompt, wherein the generating the list comprises:
deriving one or more parameters from (i) the utterance or the indication of the interface action, (ii) the data in the context variable data structure, or (iii) both, and
filling one or more input argument slots of the one or more executable actions with a version of the one or more parameters that conforms to the object schema;
executing the one or more executable actions based on the version of the one or more parameters to obtain an output; and
sending the output or a communication derived from the output to the user.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, further comprising:
identifying the flow from a plurality of flows based on (i) the utterance or the indication of the interface action, (ii) the data in the context variable data structure, or (iii) both, wherein the flow includes a flow name; and
determining whether there is an existing thread running within a session associated with the user and the application that has an identifier associated with the flow name, when the existing thread has the identifier associated with the flow name, using the existing thread as the thread for invoking the flow, and when the existing thread does not have the identifier associated with the flow name or there is no existing thread, creating a new thread and using the new thread as the thread for invoking the flow.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein:
the machine learning model is another GenAI model; and
determining the intent of the user, comprises resolving differences in intents of the user identified based on the utterance or the indication of the interface action versus the data in the context variable data structure.

### Claim 4 (depends on 3)

The computer-implemented method of claim 3, wherein when resolving the differences in the intents includes identifying an incorrect intent from page context within the data in the context variable data structure, the computer-implemented method further comprises:
triggering a navigation event on the application that causes a new page to be displayed to the user, wherein the new page is associated with a correct intent identified from resolving the differences in the intents of the user, and
updating the context variable data structure with page context associated with the new page.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, further comprising:
prior to executing the one or more executable actions, determining whether at least one of the one or more input argument slots of the one or more executable actions is unable to be filled with a version of the one or more parameters that conforms to the object schema;
responsive to determining at least one of the one or more input argument slots of the one or more executable actions is unable to be filled, generating, using another GenAI, a request for one or more additional parameters associated with the at least one of the one or more input argument slots; and
communicating the request to the user via the application.

### Claim 6 (depends on 5)

The computer-implemented method of claim 5, further comprising:
receiving a subsequent utterance from the user, the subsequent utterance comprising the one or more additional parameters; and
generating, using the GenAI model, the list comprising one or more executable actions based on the prompt, wherein the generating the list comprises:
deriving the one or more additional parameters from the subsequent utterance, and
filling the one or more input argument slots of the one or more executable actions with a version of the one or more additional parameters that conforms to the object schema.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, wherein sending the output or the communication derived from the output to the user, comprises triggering a navigation event on the application that causes a new page to be displayed to the user that is associated with the output or the communication derived from the output to the user.

### Claim 8

A system comprising:
one or more processors; and
one or more computer-readable media storing instructions which, when executed by the one or more processors, cause the system to perform operations comprising:
receiving a message payload comprising: (i) an utterance or indication of an interface action, and (ii) application context, associated with an interaction between a user and an application;
invoking, using a thread, a flow based on the message payload;
generating a context variable data structure associated with the thread, wherein generating the context variable data structure comprises storing one or more variables from the application context as at least a portion of data stored within the context variable data structure;
responsive to invoking the flow, determining, using a machine learning model, an intent of the user based on (i) the utterance or the indication of the interface action, (ii) the data in the context variable data structure, or (iii) both;
accessing, based on the intent, a prompt and an object schema;
inserting: the utterance or the indication of the interface action, at least a portion of the data in the context variable data structure, and the object schema into the prompt;
generating, using a generative artificial intelligence (GenAI) model, a list comprising one or more executable actions based on the prompt, wherein the generating the list comprises:
deriving one or more parameters from (i) the utterance or the indication of the interface action, (ii) the data in the context variable data structure, or (iii) both, and
filling one or more input argument slots of the one or more executable actions with a version of the one or more parameters that conforms to the object schema;
executing the one or more executable actions based on the version of the one or more parameters to obtain an output; and
sending the output or a communication derived from the output to the user.

### Claim 9 (depends on 8)

The system of claim 8, wherein the operations further comprise:
identifying the flow from a plurality of flows based on (i) the utterance or the indication of the interface action, (ii) the data in the context variable data structure, or (iii) both, wherein the flow includes a flow name; and
determining whether there is an existing thread running within a session associated with the user and the application that has an identifier associated with the flow name, when the existing thread has the identifier associated with the flow name, using the existing thread as the thread for invoking the flow, and when the existing thread does not have the identifier associated with the flow name or there is no existing thread, creating a new thread and using the new thread as the thread for invoking the flow.

### Claim 10 (depends on 8)

The system of claim 8, wherein:
the machine learning model is another GenAI model; and
determining the intent of the user, comprises resolving differences in intents of the user identified based on the utterance or the indication of the interface action versus the data in the context variable data structure.

### Claim 11 (depends on 10)

The system of claim 10, wherein when resolving the differences in the intents includes identifying an incorrect intent from page context within the data in the context variable data structure, the computer-implemented method further comprises:
triggering a navigation event on the application that causes a new page to be displayed to the user, wherein the new page is associated with a correct intent identified from resolving the differences in the intents of the user, and
updating the context variable data structure with page context associated with the new page.

### Claim 12 (depends on 8)

The system of claim 8, wherein the operations further comprise:
prior to executing the one or more executable actions, determining whether at least one of the one or more input argument slots of the one or more executable actions is unable to be filled with a version of the one or more parameters that conforms to the object schema;
responsive to determining at least one of the one or more input argument slots of the one or more executable actions is unable to be filled, generating, by another GenAI, a request for one or more additional parameters associated with the at least one of the one or more input argument slots; and
communicating the request to the user via the application.

### Claim 13 (depends on 12)

The system of claim 12, wherein the operations further comprise:
receiving a subsequent utterance from the user, the subsequent utterance comprising the one or more additional parameters; and
generating, by the GenAI model, the list comprising one or more executable actions based on the prompt, wherein the generating the list comprises:
deriving the one or more additional parameters from the subsequent utterance, and
filling the one or more input argument slots of the one or more executable actions with a version of the one or more additional parameters that conforms to the object schema.

### Claim 14 (depends on 8)

The system of claim 8, wherein sending the output or the communication derived from the output to the user, comprises triggering a navigation event on the application that causes a new page to be displayed to the user that is associated with the output or the communication derived from the output to the user.

### Claim 15

One or more non-transitory computer-readable media storing instructions which, when executed by one or more processors, cause the one or more processors to perform operations comprising:
receiving a message payload comprising: (i) an utterance or indication of an interface action, and (ii) application context, associated with an interaction between a user and an application;
invoking, using a thread, a flow based on the message payload;
generating a context variable data structure associated with the thread, wherein generating the context variable data structure comprises storing one or more variables from the application context as at least a portion of data stored within the context variable data structure;
responsive to invoking the flow, determining, using a machine learning model, an intent of the user based on (i) the utterance or the indication of the interface action, (ii) the data in the context variable data structure, or (iii) both;
accessing, based on the intent, a prompt and an object schema;
inserting: the utterance or the indication of the interface action, at least a portion of the data in the context variable data structure, and the object schema into the prompt;
generating, using a generative artificial intelligence (GenAI) model, a list comprising one or more executable actions based on the prompt, wherein the generating the list comprises:
deriving one or more parameters from (i) the utterance or the indication of the interface action, (ii) the data in the context variable data structure, or (iii) both, and
filling one or more input argument slots of the one or more executable actions with a version of the one or more parameters that conforms to the object schema;
executing the one or more executable actions based on the version of the one or more parameters to obtain an output; and
sending the output or a communication derived from the output to the user.

### Claim 16 (depends on 15)

The one or more non-transitory computer-readable media of claim 15, wherein the operations further comprise:
identifying the flow from a plurality of flows based on (i) the utterance or the indication of the interface action, (ii) the data in the context variable data structure, or (iii) both, wherein the flow includes a flow name; and
determining whether there is an existing thread running within a session associated with the user and the application that has an identifier associated with the flow name, when the existing thread has the identifier associated with the flow name, using the existing thread as the thread for invoking the flow, and when the existing thread does not have the identifier associated with the flow name or there is no existing thread, creating a new thread and using the new thread as the thread for invoking the flow.

### Claim 17 (depends on 15)

The one or more non-transitory computer-readable media of claim 15, wherein:
the machine learning model is another GenAI model; and
determining the intent of the user, comprises resolving differences in intents of the user identified based on the utterance or the indication of the interface action versus the data in the context variable data structure.

### Claim 18 (depends on 17)

The one or more non-transitory computer-readable media of claim 17, wherein when resolving the differences in the intents includes identifying an incorrect intent from page context within the data in the context variable data structure, the computer-implemented method further comprises:
triggering a navigation event on the application that causes a new page to be displayed to the user, wherein the new page is associated with a correct intent identified from resolving the differences in the intents of the user, and
updating the context variable data structure with page context associated with the new page.

### Claim 19 (depends on 15)

The one or more non-transitory computer-readable media of claim 15, wherein the operations further comprise:
prior to executing the one or more executable actions, determining whether at least one of the one or more input argument slots of the one or more executable actions is unable to be filled with a version of the one or more parameters that conforms to the object schema;
responsive to determining at least one of the one or more input argument slots of the one or more executable actions is unable to be filled, generating, by another GenAI, a request for one or more additional parameters associated with the at least one of the one or more input argument slots; and
communicating the request to the user via the application.

### Claim 20 (depends on 19)

The one or more non-transitory computer-readable media of claim 19, wherein the operations further comprise:
receiving a subsequent utterance from the user, the subsequent utterance comprising the one or more additional parameters; and
generating, by the GenAI model, the list comprising one or more executable actions based on the prompt, wherein the generating the list comprises:
deriving the one or more additional parameters from the subsequent utterance, and
filling the one or more input argument slots of the one or more executable actions with a version of the one or more additional parameters that conforms to the object schema.
