# US 11,114,100 B2 · Proactive incorporation of unsolicited content into human-to-computer dialogs > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,114,100 B2 - Document ID: US11114100B2 - Status: Granted - Kind: B2 - Application number: 16/549,403 - Filed: 2019-08-23 - Published: 2019-12-12 - Granted: 2021-09-07 - Earliest claimed: 2017-05-03 - Assignees: Google Inc. - Applicants: Google LLC - Inventors: Vladimir Vuskovic; Stephan Wenger; Zineb Ait Bahajji; Martin Baeuml; Alexandru Dovlecel; Gleb Skobeltsyn - Web page: https://patentgrep.com/patent/US11114100B2 ## CPC - G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue - G06F 40/295: Named entity recognition - G06F 40/35: Discourse or dialogue representation - G06F 40/56: Natural language generation - G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning} - G10L 15/222: {Barge in, i.e. overridable guidance for interrupting prompts} - G10L 2015/227: {of the speaker; Human-factor methodology} - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Methods, apparatus, and computer readable media are described related to automated assistants that proactively incorporate, into human-to-computer dialog sessions, unsolicited content of potential interest to a user. In various implementations, based on content of an existing human-to-computer dialog session between a user and an automated assistant, an entity mentioned by the user or automated assistant may be identified. Fact(s)s related to the entity or to another entity that is related to the entity may be identified based on entity data contained in database(s). For each of the fact(s), a corresponding measure of potential interest to the user may be determined. Unsolicited natural language content may then be generated that includes one or more of the facts selected based on the corresponding measure(s) of potential interest. The automated assistant may then incorporate the unsolicited content into the existing human-to-computer dialog session or a subsequent human-to-computer dialog session. ## Claims ### Claim 1 A method implemented by one or more processors, comprising: identifying, based on a request input by a user as part of a dialog session involving the user and an automated assistant executed by one or more of the processors, one or more resources that satisfy multiple criteria contained in the user's request; generating solicited natural language content, wherein the solicited natural language content conveys the one or more resources that satisfy the multiple criteria to the user; incorporating, by the automated assistant into the dialog session involving the user and the automated assistant, the solicited natural language content; receiving additional input from the user selecting one or more of the resources that satisfy the multiple criteria; proactively identifying at least one additional resource that satisfies a first criterion of the multiple criteria but fails to satisfy a second criterion of the multiple criteria; generating unsolicited natural language content, wherein the unsolicited natural language content suggests the at least one additional resource to the user as an alternative to the selected one or more resources that would require less expenditure by the user; and causing the unsolicited natural language content to be automatically output to the user without the user specifically requesting the unsolicited natural language content, wherein the unsolicited natural language output is automatically output to the user: after determining that the automated assistant has fulfilled any outstanding obligations of the automated assistant to the user in the same dialog session; or in a subsequent dialog session involving the user and the automated assistant. ### Claim 2 (depends on 1) The method of claim 1, wherein the one or more resources and the at least one additional resource comprise digital audio files. ### Claim 3 (depends on 1) The method of claim 1, wherein the one or more resources and the at least one additional resource comprise digital video files. ### Claim 4 (depends on 1) The method of claim 1, wherein the one or more resources and the at least one additional resource comprise passes. ### Claim 5 (depends on 1) The method of claim 1, wherein the second criterion comprises a time, and the at least one additional resource is associated with an earlier or later time. ### Claim 6 (depends on 1) The method of claim 1, wherein the second criterion comprises a time interval, and the at least one additional resource is associated with a different time interval that starts before, or ends after, the time interval. ### Claim 7 (depends on 1) The method of claim 1, wherein the second criterion comprises a date, and the at least one additional resource is associated with an earlier or later date. ### Claim 8 (depends on 1) The method of claim 1, wherein the request is directed by the user to the automated assistant. ### Claim 9 (depends on 1) The method of claim 1, wherein the request is directed by the user to another participant of the dialog session. ### Claim 10 A system comprising one or more processors and memory storing instructions that, in response to execution of the instructions by the one or more processors, cause the one or more processors to: identify, based on a request input by a user as part of a dialog session involving the user and an automated assistant executed by one or more of the processors, one or more resources that satisfy multiple criteria contained in the user's request; generate solicited natural language content, wherein the solicited natural language content wherein the solicited content includes one or more selectable elements corresponding to the one or more resources that satisfy the multiple criteria; incorporate, by the automated assistant into the dialog session involving the user and the automated assistant, the solicited natural language content proactively identify at least one additional resource that satisfies a first criterion of the multiple criteria but fails to satisfy a second criterion of the multiple criteria; generate unsolicited natural language content, wherein the unsolicited natural language content suggests the at least one additional resource to the user as an alternative to the one or more resources that would require less expenditure by the user; and cause the unsolicited natural language content to be automatically output to the user without the user specifically requesting the unsolicited natural language content, wherein the unsolicited natural language output is automatically output to the user: after determining that the automated assistant has fulfilled any outstanding obligations of the automated assistant to the user; or in a subsequent dialog session involving the user and the automated assistant. ### Claim 11 (depends on 10) The system of claim 10, wherein the one or more resources and the at least one additional resource comprise digital audio files. ### Claim 12 (depends on 10) The system of claim 10, wherein the one or more resources and the at least one additional resource comprise digital video files. ### Claim 13 (depends on 10) The system of claim 10, wherein the one or more resources and the at least one additional resource comprise passes. ### Claim 14 (depends on 10) The system of claim 10, wherein the second criterion comprises a time, and the at least one additional resource is associated with an earlier or later time. ### Claim 15 (depends on 10) The system of claim 10, wherein the second criterion comprises a time interval, and the at least one additional resource is associated with a different time interval that starts before, or ends after, the time interval. ### Claim 16 (depends on 10) The system of claim 10, wherein the second criterion comprises a date, and the at least one additional resource is associated with an earlier or later date. ### Claim 17 (depends on 10) The system of claim 10, wherein the request is directed by the user to the automated assistant. ### Claim 18 A method implemented by one or more processors, comprising: identifying, based on a request input by a user as part of a dialog session involving the user and an automated assistant executed by one or more of the processors, one or more resources that satisfy multiple criteria contained in the user's request; generating solicited content, wherein the solicited content includes one or more selectable elements corresponding to the one or more resources that satisfy the multiple criteria; incorporating, by the automated assistant into the dialog session involving the user and the automated assistant, the solicited content; proactively identifying at least one additional resource that satisfies a first criterion of the multiple criteria but fails to satisfy a second criterion of the multiple criteria; generating unsolicited natural language content, wherein the unsolicited natural language content suggests the at least one additional resource to the user as an alternative to the one or more resources that would require less expenditure by the user; and causing the unsolicited natural language content to be automatically output to the user without the user specifically requesting the unsolicited natural language content, wherein the unsolicited natural language output is automatically output to the user: after determining that the automated assistant has fulfilled any outstanding obligations of the automated assistant to the user in the same dialog session; or in a subsequent dialog session involving the user and the automated assistant. ### Claim 19 (depends on 18) The method of claim 18, wherein the one or more resources and the at least one additional resource comprise passes. ### Claim 20 (depends on 18) The method of claim 18, wherein the second criterion comprises a time, and the at least one additional resource is associated with an earner or later time.