# US 12,183,342 B2 · Proactive incorporation of unsolicited content into human-to-computer dialogs

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,183,342 B2
- Document ID: US12183342B2
- Status: Granted
- Kind: B2
- Application number: 18/230,581
- Filed: 2023-08-04
- Published: 2023-11-23
- Granted: 2024-12-31
- Earliest claimed: 2017-05-03
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Vladimir Vuskovic; Stephan Wenger; Zineb Ait Bahajji; Martin Baeuml; Alexandru Dovlecel; Gleb Skobeltsyn
- Web page: https://patentgrep.com/patent/US12183342B2

## CPC

- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G06F 40/295: Named entity recognition
- G06F 40/35: Discourse or dialogue representation
- G06F 40/56: Natural language generation
- G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning}
- G10L 15/222: {Barge in, i.e. overridable guidance for interrupting prompts}
- G10L 2015/227: {of the speaker; Human-factor methodology}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Methods, apparatus, and computer readable media are described related to automated assistants that proactively incorporate, into human-to-computer dialog sessions, unsolicited content of potential interest to a user. In various implementations, based on content of an existing human-to-computer dialog session between a user and an automated assistant, an entity mentioned by the user or automated assistant may be identified. Fact(s)s related to the entity or to another entity that is related to the entity may be identified based on entity data contained in database(s). For each of the fact(s), a corresponding measure of potential interest to the user may be determined. Unsolicited natural language content may then be generated that includes one or more of the facts selected based on the corresponding measure(s) of potential interest. The automated assistant may then incorporate the unsolicited content into the existing human-to-computer dialog session or a subsequent human-to-computer dialog session.

## Claims

### Claim 1

A method implemented using one or more processors, the method comprising:
processing a voice input provided by a user as part of a dialog session involving the user and an automated assistant executed by one or more of the processors;
generating solicited natural language content, wherein the solicited natural language content is responsive to a request identified in the voice input based on the processing;
incorporating, by the automated assistant into the dialog session involving the user and the automated assistant, the solicited natural language content;
identifying additional content that is tangential to the request identified in the voice input or to the solicited natural language content, wherein the additional content includes one or more facts;
determining a measure of potential interest of the user to receive one or more of the facts, wherein the measure of potential interest reflects whether one or more of the same facts has been previously presented to the user in the existing human-to-computer dialog session between the user and the automated assistant or in a previous human-to-computer dialog session between the user and the automated assistant;
in response to determining that the measure of potential interest satisfies a threshold, causing unsolicited natural language content to be automatically output to the user without the user specifically requesting the unsolicited natural language content, wherein the unsolicited natural language output incorporates the additional content; and
in response to determining that the measure of potential interest fails to satisfy the threshold, refraining from causing unsolicited natural language content to be automatically output to the user.

### Claim 2 (depends on 1)

The method of claim 1, wherein the measure of potential interest is further determined based on contextual information associated with the user, wherein the contextual information includes traffic detected near a current location of the user or an accelerometer signal generated by a computing device carried by the user.

### Claim 3 (depends on 1)

The method of claim 1, wherein the measure of potential interest is further determined based on contextual information associated with the user, wherein the contextual information includes past human-to-computer dialogs between the user and the automated assistant or sentiment analysis of speech recognition output of the voice input.

### Claim 4 (depends on 1)

The method of claim 1, wherein the measure of potential interest is further determined based on contextual information associated with the user, wherein the contextual information includes one or more applications currently being interacted with by the user or a state of an application operating on a computing device controlled by the user.

### Claim 5 (depends on 1)

The method of claim 1, wherein one or more of the facts are selected based on one or more entities mentioned in the request identified in the voice input.

### Claim 6 (depends on 1)

The method of claim 1, wherein one or more of the facts are selected based on one or more entities mentioned in the solicited natural language content.

### Claim 7 (depends on 1)

The method of claim 1, wherein the additional content comprises a query that is tangential to the request identified in the voice input.

### Claim 8 (depends on 1)

The method of claim 1, wherein the additional content comprises a query that is tangential to the solicited natural language content.

### Claim 9

A system comprising one or more processors and memory storing instructions that, in response to execution by the one or more processors, cause the one or more processors to:
process a voice input provided by a user as part of a dialog session involving the user and an automated assistant executed by one or more of the processors;
generate solicited natural language content, wherein the solicited natural language content is responsive to a request identified in the voice input based on the processing;
incorporate, by the automated assistant into the dialog session involving the user and the automated assistant, the solicited natural language content;
identify additional content that is tangential to the request identified in the voice input or to the solicited natural language content, wherein the additional content includes one or more facts;
determine a measure of potential interest desirability of the user to receive one or more of the facts, wherein the measure of potential interest reflects whether one or more of the same facts has been previously presented to the user in the existing human-to-computer dialog session between the user and the automated assistant or in a previous human-to-computer dialog session between the user and the automated assistant;
in response to a determination that the measure of potential interest satisfies a threshold, cause unsolicited natural language content to be automatically output to the user without the user specifically requesting the unsolicited natural language content, wherein the unsolicited natural language output incorporates the additional content; and
in response to a determination that the measure of potential interest fails to satisfy the threshold, refrain from causing unsolicited natural language content to be automatically output to the user.

### Claim 10 (depends on 9)

The system of claim 9, wherein the measure of potential interest is further determined based on contextual information associated with the user, wherein the contextual information includes traffic detected near a current location of the user or an accelerometer signal generated by a computing device carried by the user.

### Claim 11 (depends on 9)

The system of claim 9, wherein the measure of potential interest is further determined based on contextual information associated with the user, wherein the contextual information includes past human-to-computer dialogs between the user and the automated assistant or sentiment analysis of speech recognition output of the voice input.

### Claim 12 (depends on 9)

The system of claim 9, wherein the measure of potential interest is further determined based on contextual information associated with the user, wherein the contextual information includes one or more applications currently being interacted with by the user or a state of an application operating on a computing device controlled by the user.

### Claim 13 (depends on 9)

The system of claim 9, wherein one or more of the facts are selected based on one or more entities mentioned in the request identified in the voice input.

### Claim 14 (depends on 9)

The system of claim 9, wherein one or more of the facts are selected based on one or more entities mentioned in the solicited natural language content.

### Claim 15 (depends on 9)

The system of claim 9, wherein the additional content comprises a query that is tangential to the request identified in the voice input.

### Claim 16 (depends on 9)

The system of claim 9, wherein the additional content comprises a query that is tangential to the solicited natural language content.

### Claim 17

At least one non-transitory computer-readable medium comprising instructions that, in response to execution by one or more processors, cause the one or more processors to:
process a voice input provided by a user as part of a dialog session involving the user and an automated assistant executed by one or more of the processors;
generate solicited natural language content, wherein the solicited natural language content is responsive to a request identified in the voice input based on the processing;
incorporate, by the automated assistant into the dialog session involving the user and the automated assistant, the solicited natural language content;
identify additional content that is tangential to the request identified in the voice input or to the solicited natural language content, wherein the additional content includes one or more facts;
determine a measure of potential interest of the user to receive one or more of the facts, wherein the measure of potential interest reflects whether one or more of the same facts has been previously presented to the user in the existing human-to-computer dialog session between the user and the automated assistant or in a previous human-to-computer dialog session between the user and the automated assistant;
in response to a determination that the measure of potential interest satisfies a threshold, cause unsolicited natural language content to be automatically output to the user without the user specifically requesting the unsolicited natural language content, wherein the unsolicited natural language output incorporates the additional content; and
in response to a determination that the measure of potential interest fails to satisfy the threshold, refrain from causing unsolicited natural language content to be automatically output to the user.

### Claim 18 (depends on 17)

The at least one non-transitory computer-readable medium of claim 17, wherein the measure of potential interest is further determined based on contextual information associated with the user, wherein the contextual information includes traffic detected near a current location of the user or an accelerometer signal generated by a computing device carried by the user.

### Claim 19 (depends on 17)

The at least one non-transitory computer-readable medium of claim 17, wherein the measure of potential interest is further determined based on contextual information associated with the user, wherein the contextual information includes past human-to-computer dialogs between the user and the automated assistant or sentiment analysis of speech recognition output of the voice input.

### Claim 20 (depends on 17)

The at least one non-transitory computer-readable medium of claim 17, wherein the measure of potential interest is further determined based on contextual information associated with the user, wherein the contextual information includes one or more applications currently being interacted with by the user or a state of an application operating on a computing device controlled by the user.