# US 2021/0279085 A1 · Ai-driven human-computer interface for associating low-level content with high-level activities using topics as an abstraction

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0279085 A1
- Document ID: US20210279085A1
- Status: Published application
- Kind: A1
- Application number: 17/242,214
- Filed: 2021-04-27
- Published: 2021-09-09
- Earliest claimed: 2018-06-27
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Aniruddha Prabhakar KULKARNI; Nathaniel M. MYHRE; Yogesh Madhukarrao JOSHI; William Henry GATES, III; Vignesh SACHIDANANDAM; Peter Loren ENGRAV
- Web page: https://patentgrep.com/patent/US20210279085A1

## CPC

- G06F 9/453: {Help systems}
- G06Q 10/107: Computer-aided management of electronic mailing [e-mailing]
- G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour
- G06F 16/90335: {Query processing}
- G06Q 10/06: Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06N 5/04: Inference or reasoning models
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

An artificial intelligence (“AI”) based system is disclosed for associating low-level user content, such as documents, email messages, and calendar invites, with high-level user activities using topics as an abstraction. The associations can enable a computing system to provide, among other things, activity-specific views that present a specific selection of low-level user content that is most relevant to a user at a particular point in time. The activity-specific views present the right information to users at the right time based on a context of a user and a user's past activities.

## Claims

### Claim at position 1

1-20. (canceled)

### Claim 21

A computer-implemented method, comprising:
receiving, from one or more data sources, content associated with a user;
receiving, via a configuration user interface (UI), first user input that defines a first activity identifier that indicates a first activity that is engaged in by the user;
receiving, via the configuration UI, second user input that defines a second activity identifier that indicates a second activity that is engaged in by the user;
based on the first user input and the second user input, causing an artificial intelligence (AI) engine to analyze the content based on an AI model to determine:
a first topic, from the content associated with the user, that is associated with the first activity that is engaged in by the user, and
a second topic, from the content associated with the user, that is associated with the second activity that is engaged in by the user;
rendering, via the configuration UI, at least:
a first topic identifier UI element that identifies the first topic determined to be associated with the first activity that is engaged in by the user, and
a second topic identifier UI element that identifies the second topic determined to be associated with the second activity that is engaged in by the user;
generating an activity graph that is associated with the user and that includes:
a first node that corresponds to the first activity,
a second node that corresponds to the second activity,
a third node that corresponds to the first topic and that is linked, within the activity graph, to the first node that corresponds to the first activity, and
a fourth node that corresponds to the second topic and that is linked, within the activity graph, to the second node that corresponds to the second activity;
receiving, via the configuration UI, third user input associated with the first topic identifier UI element, the third user input confirming that the first topic is associated with the first activity that is engaged in by the user;
receiving, via the configuration UI, fourth user input associated with the second topic identifier UI element, the fourth user input refuting that the second topic is associated with the second activity that is engaged in by the user; and
updating the AI model and the activity graph based on the fourth user input refuting that the second topic is associated with the second activity that is engaged in by the user.

### Claim 22 (depends on 21)

The computer-implemented method of claim 21, further comprising providing data describing an association of the first topic identified by the first topic identifier UI element to the AI engine for updating an AI model used by the AI engine to identify one or more topics associated with the first activity identifier.

### Claim 23 (depends on 21)

The computer-implemented method of claim 21, wherein the first topic identifier UI element is sized according to a volume of the content associated with the first topic.

### Claim 24 (depends on 21)

The computer-implemented method of claim 21, wherein the first topic identifier UI element is sized according to a relevance of the first topic.

### Claim 25 (depends on 24)

The computer-implemented method of claim 24, further comprising:
receiving fifth user input that resizes the first topic identifier UI element; and
responsive to receiving the fifth user input, providing data to the AI engine for reducing or increasing the relevance of the first topic in the AI model.

### Claim 26 (depends on 21)

The computer-implemented method of claim 21, wherein the first topic identifier UI element is ordered, in relation to one or more other topic identifier UI elements, according to the relevance of the first topic in relation to one or more other topics that are individually associated with the one or more other topic identifier UI elements.

### Claim 27 (depends on 26)

The computer-implemented method of claim 26, further comprising:
receiving fifth user input that reorders the one or more other topic identifier UI elements; and
responsive to receiving the fifth user input, providing data to the AI engine for modifying the relevance of the reordered topic identifier UI elements in the AI model.

### Claim 28

A computing system, comprising:
one or more processors; and
a computer storage medium having computer-executable instructions stored thereupon which, when executed by the one or more processors, cause the computing system to:
receive, from one or more data sources, content associated with a user;
receive, via a configuration user interface (UI), first user input that defines a first activity identifier that indicates a first activity that is engaged in by the user;
receive, via the configuration UI, second user input that defines a second activity identifier that indicates a second activity that is engaged in by the user;
based on the first user input and the second user input, cause an artificial intelligence (AI) engine to analyze the content based on an AI model to determine:
a first topic, from the content associated with the user, that is associated with the first activity that is engaged in by the user, and
a second topic, from the content associated with the user, that is associated with the second activity that is engaged in by the user;
render, via the configuration UI, at least:
a first topic identifier UI element that identifies the first topic determined to be associated with the first activity that is engaged in by the user, and
a second topic identifier UI element that identifies the second topic determined to be associated with the second activity that is engaged in by the user;
generate an activity graph that is associated with the user and that includes:
a first node that corresponds to the first activity,
a second node that corresponds to the second activity,
a third node that corresponds to the first topic and that is linked, within the activity graph, to the first node that corresponds to the first activity, and
a fourth node that corresponds to the second topic and that is linked, within the activity graph, to the second node that corresponds to the second activity;
receive, via the configuration UI, third user input associated with the first topic identifier UI element, the third user input confirming that the first topic is associated with the first activity that is engaged in by the user;
receive, via the configuration, fourth user input associated with the second topic identifier UI element, the fourth user input refuting that the second topic is associated with the second activity that is engaged in by the user; and
update the AI model and the activity graph based on the fourth user input refuting that the second topic is associated with the second activity that is engaged in by the user.

### Claim 29 (depends on 28)

The computing system of claim 28, wherein the computer storage medium has further computer-executable instructions stored thereupon which, when executed by the one or more processors, cause the computing system to provide data describing an association of the first topic identified by the first topic identifier UI element with the first activity to the AI engine for updating the AI model used to identify one of more topics associated with the first activity.

### Claim 30 (depends on 28)

The computing system of claim 28, wherein the first topic identifier UI element is sized according to a relevance of the first topic in relation to one or more other topics.

### Claim 31 (depends on 30)

The computing system of claim 30, wherein the computer storage medium has further computer-executable instructions stored thereupon which, when executed by the one or more processors, cause the computing system to:
receive fifth user input that resizes the first topic identifier UI element; and
responsive to receiving the fifth user input, provide data to the AI engine for reducing or increasing the relevance of the first topic in the AI model.

### Claim 32 (depends on 28)

The computing system of claim 28, wherein the computer storage medium has further computer-executable instructions stored thereupon which, when executed by the one or more processors, cause the computing system to:
receive a selection of the first topic identifier UI element; and
responsive to receiving the selection, present data in the configuration UI that identifies instances of the content associated with the first topic.

### Claim 33 (depends on 32)

The computing system of claim 32, wherein the computer storage medium has further computer-executable instructions stored thereupon which, when executed by the one or more processors, cause the computing system to:
receive fifth user input that associates an instance of the content with the first topic identifier UI element; and
responsive to receiving the fifth user input, providing data describing the association of the instance of the content with the first topic to the AI engine for updating the AI model.

### Claim 34 (depends on 28)

The computing system of claim 28, wherein the first topic identifier UI element comprises a photo of a person associated with the first topic.

### Claim 35

A computer storage medium having computer-executable instructions stored thereupon which, when executed by one or more processors of a computing system, cause the computing system to:
receive, from one or more data sources, content associated with a user;
receive, via a configuration user interface (UI), first user input that defines a first activity identifier that indicates a first activity that is engaged in by a user;
receive, via the configuration UI, second user input that defines a second activity identifier that indicates a second activity that is engaged in by the user;
based on the first user input and the second user input, cause an artificial intelligence (AI) engine to analyze the content based on an AI model to determine:
a first topic, from the content associated with the user, that is associated with the first activity that is engaged in by the user, and
a second topic, from the content associated with the user, that is associated with the second activity that is engaged in by the user;
generate an activity graph that is associated with the user and that includes:
a first node that corresponds to the first activity,
a second node that corresponds to the second activity,
a third node that corresponds to the first topic and that is linked, within the activity graph, to the first node that corresponds to the first activity, and
a fourth node that corresponds to the second topic and that is linked, within the activity graph, to the second node that corresponds to the second activity;
cause at least a portion of the activity graph to be rendered via the configuration UI;
receive, via the configuration UI that renders the at least the portion of the activity graph, third user input confirming that the first topic is associated with the first activity that is engaged in by the user;
receive, via the configuration UI that renders the at least the portion of the activity graph, fourth user input refuting that the second topic is associated with the second activity that is engaged in by the user; and
update the AI model and the activity graph based on the fourth user input refuting that the second topic is associated with the second activity that is engaged in by the user.

### Claim 36 (depends on 35)

The computer storage medium of claim 35, wherein the computer storage medium has further computer-executable instructions stored thereupon which, when executed by the one or more processors of the computing system, cause the computing system to:
render, via the configuration UI, a first topic identifier UI element that identifies the first topic determined to be associated with the first activity that is engaged in by the user; and
render, via the configuration UI, a second topic identifier UI element that identifies the second topic determined to be associated with the second activity that is engaged in by the user.

### Claim 37 (depends on 36)

The computer storage medium of claim 36, wherein the first topic identifier UI element and the second topic identifier UI element are sized according to a relevance of the first topic in relation to a relevance of the second topic.

### Claim 38 (depends on 36)

The computer storage medium of claim 36, wherein the first topic identifier UI element and the second topic identifier UI element are ordered according to a relevance of the first topic in relation to a relevance of the second topic.

### Claim 39 (depends on 36)

The computer storage medium of claim 36, wherein the first topic identifier UI element and the second topic identifier UI element are sized according to a relative volume of the content associated with the first topic in relation to the second topic.

### Claim 40 (depends on 36)

The computer storage medium of claim 36, wherein the first topic identifier UI element comprises a photo of a person associated with the first topic.
