# US 11,908,445 B2 · Conversation-aware proactive notifications for a voice interface device > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,908,445 B2 - Document ID: US11908445B2 - Status: Granted - Kind: B2 - Application number: 17/745,236 - Filed: 2022-05-16 - Published: 2022-09-01 - Granted: 2024-02-20 - Earliest claimed: 2016-12-30 - Assignees: Google Inc. - Applicants: Google LLC - Inventors: Kenneth Mixter; Daniel Colish; Tuan Nguyen - Web page: https://patentgrep.com/patent/US11908445B2 ## CPC - G10L 13/00: Speech synthesis; Text to speech systems - G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback} - G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue - G10L 15/26: Speech to text systems (G10L15/08 takes precedence) - H04L 12/282: {based on user interaction within the home (receiver circuitry for displaying additional information being controlled by a remote control apparatus H04N21/42204)} - H04L 51/224: providing notification on incoming messages, e.g. pushed notifications of received messages - H04L 67/55: Push-based network services - G10L 2015/223: {Execution procedure of a spoken command} - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract A method for proactive notifications in a voice interface device includes: receiving a first user voice request for an action with an future performance time; assigning the first user voice request to a voice assistant service for performance; subsequent to the receiving, receiving a second user voice request and in response to the second user voice request initiating a conversation with the user; and during the conversation: receiving a notification from the voice assistant service of performance of the action; triggering a first audible announcement to the user to indicate a transition from the conversation and interrupting the conversation; triggering a second audible announcement to the user to indicate performance of the action; and triggering a third audible announcement to the user to indicate a transition back to the conversation and rejoining the conversation. ## Claims ### Claim 1 A method comprising: receiving a first request, wherein an electronic device comprises a speaker, one or more processors, and memory-storing instructions for execution by the one or more processors; transmitting the first request to a first server-implemented voice assistant service via the Internet; receiving, subsequent to receiving the first request, a second request; transmitting the second request to a second server-implemented voice assistant service via the Internet; subsequent to the receiving of the second request: receiving, from the first server-implemented voice-assistant service, via the Internet, a response to the first request; receiving from the second server-implemented voice-assistant service, via the Internet, a response to the second request as part of an interaction with one or more users of the electronic device; triggering output of an audible announcement via the speaker, the audible announcement indicating a transition from the interaction with the one or more users of the electronic device; and triggering output of a verbal announcement via the speaker, the verbal announcement including the response to the first request. ### Claim 2 (depends on 1) The method of claim 1, further comprising assigning the first request to the first server-implemented voice-assistant service and assigning the second request to the second server-implemented voice-assistant service. ### Claim 3 (depends on 1) The method of claim 1, further comprising following the triggering of the verbal announcement including the response to the first request, triggering a second audible announcement indicating a transition back to the interaction with the one or more users of the electronic device. ### Claim 4 (depends on 3) The method of claim 3, further comprising following the triggering of the second audible announcement indicating the transition back to the interaction with the one or more users of the electronic device, triggering a second verbal announcement related to the interaction with the one or more users of the electronic device including the response to the second request. ### Claim 5 (depends on 1) The method of claim 1, wherein: the first request is a request for an action with a future performance time; the response to the first request includes a notification indicating performance of the action; the second request is a request for information; the response to the second request includes the requested information; and the electronic device receives the second request prior to the future performance time. ### Claim 6 (depends on 1) The method of claim 1, further comprising determining, based on context of the interaction with the one or more users of the electronic device, an appropriate time at which to trigger the audible announcement indicating the transition from the interaction. ### Claim 7 (depends on 1) The method of claim 1, wherein: the interaction with the one or more users of the electronic device is performed between a user of the electronic device and a software agent; and the software agent determines and generates the audible announcement indicating a transition from the interaction with the one or more users of the electronic device. ### Claim 8 (depends on 7) The method of claim 7, wherein: the software agent performs operations related to satisfaction of the second request; and an agent module manages interactions between the software agent and the user during the interaction with the one or more users of the electronic device. ### Claim 9 (depends on 8) The method of claim 8, wherein: the agent module comprises a library of transition-in phrases and a library of transition-out phrases; the agent module generates content of the audible announcement indicating a transition from the interaction from one or more of the transition-in phrases; and the agent module generates content of the audible announcement indicating a transition back to the interaction from one or more of the transition-out phrases. ### Claim 10 (depends on 8) The method of claim 8, wherein a conversation manager module causes the agent module to: interrupt the interaction; trigger the audible announcement indicating a transition from the interaction; wait for the conversation manager module to trigger the verbal announcement including the response to the first request; trigger the audible announcement indicating a transition back to the interaction upon completion of the verbal announcement including the response to the first request; and rejoin the interaction. ### Claim 11 An electronic device comprising a speaker, one or more processors, and memory storing executable instructions that are configured to cause the one or more processors to perform operations including: receiving a first request; transmitting the first request to a first server-implemented voice assistant service via the Internet; receiving, subsequent to receiving the first request, a second request; transmitting the second request to a second server-implemented voice assistant service via the Internet; subsequent to the receiving of the second request: receiving, from the first server-implemented voice-assistant service, via the Internet, a response to the first request; receiving from the second server-implemented voice-assistant service, via the Internet, a response to the second request as part of an interaction with one or more users of the electronic device; triggering output of an audible announcement via the speaker, the audible announcement indicating a transition from the interaction with the one or more users of the electronic device; and triggering output of a verbal announcement via the speaker, the verbal announcement including the response to the first request. ### Claim 12 (depends on 11) The electronic device of claim 11, wherein the instructions are further configured to cause the one or more processors to assign the first request to the first server-implemented voice-assistant service and assigning the second request to the second server-implemented voice-assistant service. ### Claim 13 (depends on 11) The electronic device of claim 11, wherein the instructions are further configured to cause the one or more processors to, following the triggering of the verbal announcement including the response to the first request, trigger a second audible announcement indicating a transition back to the interaction with the one or more users of the electronic device. ### Claim 14 (depends on 13) The electronic device of claim 13, wherein the instructions are further configured to cause the one or more processors to, following the triggering of the second audible announcement indicating the transition back to the interaction with the one or more users of the electronic device, trigger a second verbal announcement related to the interaction with the one or more users of the electronic device including the response to the second request. ### Claim 15 (depends on 11) The electronic device of claim 11, wherein: the first request is a request for an action with a future performance time; the response to the first request includes a notification indicating performance of the action; the second request is a request for information; the response to the second request includes the requested information; and the electronic device receives the second request prior to the future performance time. ### Claim 16 (depends on 11) The electronic device of claim 11, wherein the instructions are further configured to cause the one or more processors to determine, based on context of the interaction with the one or more users of the electronic device, an appropriate time at which to trigger the audible announcement indicating the transition from the interaction. ### Claim 17 (depends on 11) The electronic device of claim 11, wherein: the interaction with the one or more users of the electronic device is performed between a user of the electronic device and a software agent; and the software agent determines and generates the audible announcement indicating a transition from the interaction with the one or more users of the electronic device. ### Claim 18 (depends on 17) The electronic device of claim 17, wherein: the software agent performs operations related to satisfaction of the second request; and an agent module manages interactions between the software agent and the user during the interaction with the one or more users of the electronic device. ### Claim 19 (depends on 18) The electronic device of claim 18, wherein: the agent module comprises a library of transition-in phrases and a library of transition-out phrases; the agent module generates content of the audible announcement indicating a transition from the interaction from one or more of the transition-in phrases; and the agent module generates content of the audible announcement indicating a transition back to the interaction from one or more of the transition-out phrases. ### Claim 20 (depends on 18) The electronic device of claim 18, wherein a conversation manager module causes the agent module to: interrupt the interaction; trigger the audible announcement indicating a transition from the interaction; wait for the conversation manager module to trigger the verbal announcement including the response to the first request; trigger the audible announcement indicating a transition back to the interaction upon completion of the verbal announcement including the response to the first request; and rejoin the interaction. ### Claim 21 A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which, when executed by an electronic device with one or more processors, cause the one or more processors to perform: receiving a first request; transmitting the first request to a first server-implemented voice assistant service via the Internet; receiving, subsequent to receiving the first request, a second request; transmitting the second request to a second server-implemented voice assistant service via the Internet; subsequent to the receiving of the second request: receiving, from the first server-implemented voice-assistant service a response to the first request; receiving from the second server-implemented voice-assistant service a response to the second request as part of an interaction with one or more users of the electronic device; triggering an audible announcement via a speaker of the electronic device, the audible announcement indicating a transition from the interaction with the one or more users of the electronic device; and triggering a verbal announcement via the speaker, the verbal announcement including the response to the first request.