US 2026/0228225 A1 · MACHINE LEARNING TECHNIQUES FOR IMPROVING CONTEXT AND UNDERSTANDING OF USER INTERACTION-BASED DATA

Abstract

Techniques for using machine learning (ML) model(s) to generate a textual summary of a stream of events at least in part by receiving information corresponding to the stream of events corresponding to interactions between a user and application program(s) executing on a computing device, the information comprising, for each of multiple events in the stream of events associated with a process, metadata associated with the event, wherein metadata for a particular event specifies values for attributes of the particular event; processing, using at least one ML model, the metadata associated with the multiple events to generate multiple corresponding textual summaries of the multiple events; generating, using the textual summaries of the multiple events, a textual summary of the process performed by the user through the interactions between the user and the application program(s) executing on the computing device; and outputting the textual summary of the process.

Claims21

Dependent claims

-20. (canceled)

A method of generating a textual summary of events in a stream of events corresponding to interactions between a user performing a process and one or more application programs executing on a computing device, the method comprising:

using at least one computer hardware processor to perform:
receiving information corresponding to the stream of events corresponding to the interactions between the user and the one or more application programs executing on the computing device, the information comprising, for each of multiple events in the stream of events associated with the process, metadata associated with the event including first metadata associated with a first event corresponding to an interaction between the user and an application program of the one or more application programs, wherein the first metadata comprises values for one or more attributes selected from the group consisting of:
a name of the application program, a title of an application program screen of the application program with which the user interacted during the first event, an identifier of a user interface element of the application program screen with which the user interacted, a type of the user interface element of the application program screen with which the user interacted, one or more identifiers for one or more user interface elements of the application program screen with which the user did not interact, a duration of the interaction, and one or more textual phrases and/or sentences appearing on the application program screen;
processing, using at least one machine learning (ML) model, the metadata associated with the multiple events in the stream of events to generate multiple corresponding textual summaries of the multiple events, the processing comprising:
for each particular event of the multiple events in the stream of events:
processing metadata associated with the particular event using the at least one ML model to determine a business object, an activity, an intent, and/or a reason associated with the particular event;
and
generating a textual summary of the particular event from the determined business object, activity, intent, and/or reason associated with the particular event;
and
outputting the generated textual summaries of the multiple events.

7 more elements

A system, comprising:

at least one computer hardware processor;
and
at least one non-transitory computer-readable storage medium storing processor executable instructions that, when executed by the at least one computer hardware processor, cause the at least one computer hardware processor to perform a method of generating a textual summary of events in a stream of events corresponding to interactions between a user performing a process and one or more application programs executing on a computing device, the method comprising:
receiving information corresponding to the stream of events corresponding to the interactions between the user and the one or more application programs executing on the computing device, the information comprising, for each of multiple events in the stream of events associated with the process, metadata associated with the event including first metadata associated with a first event corresponding to an interaction between the user and an application program of the one or more application programs, wherein the first metadata comprises values for one or more attributes selected from the group consisting of:
a name of the application program, a title of an application program screen of the application program with which the user interacted during the first event, an identifier of a user interface element of the application program screen with which the user interacted, a type of the user interface element of the application program screen with which the user interacted, one or more identifiers for one or more user interface elements of the application program screen with which the user did not interact, a duration of the interaction, and one or more textual phrases and/or sentences appearing on the application program screen;
processing, using at least one machine learning (ML) model, the metadata associated with the multiple events in the stream of events to generate multiple corresponding textual summaries of the multiple events, the processing comprising:
for each particular event of the multiple events in the stream of events:
processing metadata associated with the particular event using the at least one ML model to determine a business object, an activity, an intent, and/or a reason associated with the particular event;
and
generating a textual summary of the particular event from the determined business object, activity, intent, and/or reason associated with the particular event;
and
outputting the generated textual summaries of the multiple events.

9 more elements

At least one non-transitory computer-readable storage medium storing processor executable instructions that, when executed by at least one computer hardware processor, cause the at least one computer hardware processor to perform a method of generating a textual summary of events in a stream of events corresponding to interactions between a user performing a process and one or more application programs executing on a computing device, the method comprising:

receiving information corresponding to the stream of events corresponding to the interactions between the user and the one or more application programs executing on the computing device, the information comprising, for each of multiple events in the stream of events associated with the process, metadata associated with the event including first metadata associated with a first event corresponding to an interaction between the user and an application program of the one or more application programs, wherein the first metadata comprises values for one or more attributes selected from the group consisting of:
a name of the application program, a title of an application program screen of the application program with which the user interacted during the first event, an identifier of a user interface element of the application program screen with which the user interacted, a type of the user interface element of the application program screen with which the user interacted, one or more identifiers for one or more user interface elements of the application program screen with which the user did not interact, a duration of the interaction, and one or more textual phrases and/or sentences appearing on the application program screen;
processing, using at least one machine learning (ML) model, the metadata associated with the multiple events in the stream of events to generate multiple corresponding textual summaries of the multiple events, the processing comprising:
for each particular event of the multiple events in the stream of events:
processing metadata associated with the particular event using the at least one ML model to determine a business object, an activity, an intent, and/or a reason associated with the particular event;
and
generating a textual summary of the particular event from the determined business object, activity, intent, and/or reason associated with the particular event;
and
outputting the generated textual summaries of the multiple events.

6 more elements

Dependencies are inferred from the claim text.

Machine learning techniques for improving context and understanding of user interaction-based data

Life of this patent

Machine learning techniques for improving context and understanding of user interaction-based data

Life of this patent

Abstract

Claims21