# US 10,482,882 B2 · Proactive incorporation of unsolicited content into human-to-computer dialogs > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,482,882 B2 - Document ID: US10482882B2 - Status: Granted - Kind: B2 - Application number: 15/825,919 - Filed: 2017-11-29 - Published: 2018-11-08 - Granted: 2019-11-19 - Earliest claimed: 2017-05-03 - Assignees: Google Inc. - Applicants: Google LLC - Inventors: Vladimir Vuskovic; Stephan Wenger; Zineb Ait Bahajji; Martin Baeuml; Alexandru Dovlecel; Gleb Skobeltsyn - Web page: https://patentgrep.com/patent/US10482882B2 ## CPC - G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue - G06F 40/295: Named entity recognition - G06F 40/35: Discourse or dialogue representation - G06F 40/56: Natural language generation - G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning} - G10L 15/222: {Barge in, i.e. overridable guidance for interrupting prompts} - G10L 2015/227: {of the speaker; Human-factor methodology} - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Methods, apparatus, and computer readable media are described related to automated assistants that proactively incorporate, into human-to-computer dialog sessions, unsolicited content of potential interest to a user. In various implementations, based on content of an existing human-to-computer dialog session between a user and an automated assistant, an entity mentioned by the user or automated assistant may be identified. Fact(s)s related to the entity or to another entity that is related to the entity may be identified based on entity data contained in database(s). For each of the fact(s), a corresponding measure of potential interest to the user may be determined. Unsolicited natural language content may then be generated that includes one or more of the facts selected based on the corresponding measure(s) of potential interest. The automated assistant may then incorporate the unsolicited content into the existing human-to-computer dialog session or a subsequent human-to-computer dialog session. ## Claims ### Claim 1 A method implemented by one or more processors, comprising: identifying an entity based on a state of a media or gaming software application being executed on a first client device operated by a user, wherein the entity is identified without using explicit input from the user; determining that an automated assistant operating on a second client device associated with the user has no outstanding obligations to the user, wherein the first and second client devices are part of a coordinated ecosystem of client devices; identifying one or more facts about the entity based on entity data contained in one or more databases; determining, for each of the one or more facts, a corresponding measure of potential interest to the user; generating, by one or more of the processors, unsolicited natural language content, wherein the unsolicited natural language content includes one or more of the facts selected based on the corresponding one or more measures of potential interest; and after the determination that the automated assistant has no outstanding obligations to the user, incorporating, by the automated assistant into a new or existing human-to-computer dialog session between the user and the automated assistant, the unsolicited natural language content, wherein the incorporating causes the unsolicited natural language content to be automatically output to the user as part of the new or existing human-to-computer dialog session. ### Claim 2 (depends on 1) The method of claim 1, wherein determining the corresponding measure of potential interest is based on data associated with a user profile associated the user. ### Claim 3 (depends on 2) The method of claim 2, wherein the data associated with the user profile includes a search history associated with the user. ### Claim 4 (depends on 1) The method of claim 1, wherein determining the corresponding measure of potential interest is based on analysis of a textual corpus of one or more previous online conversations between multiple people different than the user. ### Claim 5 (depends on 4) The method of claim 4, wherein the analysis includes detection in the textual corpus of one or more references to one or more entities, as well as detection of one or references to facts about the one or more entities in the textual corpus within a particular proximity of the one or more references to the one or more entities. ### Claim 6 (depends on 5) The method of claim 5, wherein the one or more entities include the entity determined based on the state of the media or gaming software application. ### Claim 7 (depends on 5) The method of claim 5, wherein the one or more entities share an entity class with the entity determined based on the state of the media or gaming software application. ### Claim 8 (depends on 5) The method of claim 5, wherein the one or more entities share one or more attributes with the entity determined based on the state of the media or gaming software application. ### Claim 9 (depends on 1) The method of claim 1, wherein determining the corresponding measure of potential interest comprises detecting that a given fact of the one or more facts has been previously referenced in the existing human-to-computer dialog session between the user and the automated assistant or in a previous human-to-computer dialog session between the user and the automated assistant, wherein the measure of potential interest determined for the given fact reflects the detecting. ### Claim 10 (depends on 1) The method of claim 1, further comprising eliminating, by one or more of the processors, a given fact of the one or more facts from consideration based on detecting that the given fact has been previously referenced in the existing human-to-computer dialog session between the user and the automated assistant or in a previous human-to-computer dialog session between the user and the automated assistant. ### Claim 11 (depends on 1) The method of claim 1, wherein the entity relates to a video game, and a given fact of the one or more facts is a fact related to the video game. ### Claim 12 (depends on 1) The method of claim 1, wherein the entity is a person, and a given fact of the one or more facts is an upcoming event involving the person. ### Claim 13 A system comprising one or more processors and memory operably coupled with the one or more processors, wherein the memory stores instructions that, in response to execution of the instructions by one or more processors, cause the one or more processors to perform the following operations: identifying an entity based on a state of a media or gaming software application executing on a first client device operated by a user, wherein the entity is identified without using explicit input from the user; determining that an automated assistant operating on a second client device associated with the user has no outstanding obligations to the user, wherein the first and second client devices are part of a coordinated ecosystem of client devices; identifying one or more facts about the entity based on entity data contained in one or more databases; determining, for each of the one or more facts, a corresponding measure of potential interest to the user; generating, by one or more of the processors, unsolicited natural language content, wherein the unsolicited natural language content includes one or more of the facts selected based on the corresponding one or more measures of potential interest; and after the determination that the automated assistant has no outstanding obligations to the user, incorporating, by the automated assistant into a new or existing human-to-computer dialog session between the user and the automated assistant, the unsolicited natural language content, wherein the incorporating causes the unsolicited natural language content to be automatically output to the user as part of the new or existing human-to-computer dialog session. ### Claim 14 (depends on 13) The system of claim 13, wherein determining the corresponding measure of potential interest is based on data associated with a user profile associated the user. ### Claim 15 (depends on 14) The system of claim 14, wherein the data associated with the user profile includes a search history associated with the user. ### Claim 16 (depends on 13) The system of claim 13, wherein determining the corresponding measure of potential interest is based on analysis of a textual corpus of one or more previous online conversations between multiple people different than the user. ### Claim 17 (depends on 16) The system of claim 16, wherein the analysis includes detection in the textual corpus of one or more references to one or more entities, as well as detection of one or references to facts about the one or more entities in the textual corpus within a particular proximity of the one or more references to the one or more entities. ### Claim 18 (depends on 17) The system of claim 17, wherein the one or more entities include the entity determined based on the state of the media or gaming software application. ### Claim 19 (depends on 13) The system of claim 13, wherein the entity relates to a video game, and a given fact of the one or more facts is a fact related to the video game. ### Claim 20 At least one non-transitory computer-readable medium comprising instructions that, in response to execution of the instructions by one or more processors, cause the one or more processors to perform the following operations: identifying an entity based on a state of a media or gaming software application executing on a first client device operated by a user, wherein the entity is identified without using explicit input from the user; determining that an automated assistant operating on a second client device associated with the user has no outstanding obligations to the user, wherein the first and second client devices are part of a coordinated ecosystem of client devices; identifying one or more facts about the entity based on entity data contained in one or more databases; determining, for each of the one or more facts, a corresponding measure of potential interest to the user; generating, by one or more of the processors, unsolicited natural language content, wherein the unsolicited natural language content includes one or more of the facts selected based on the corresponding one or more measures of potential interest; and after the determination that the automated assistant has no outstanding obligations to the user, incorporating, by the automated assistant into a new or existing human-to-computer dialog session between the user and the automated assistant, the unsolicited natural language content, wherein the incorporating causes the unsolicited natural language content to be automatically output to the user as part of the new or existing human-to-computer dialog session.