# US 12,112,755 B2 · Automated calling system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,112,755 B2 - Document ID: US12112755B2 - Status: Granted - Kind: B2 - Application number: 17/940,732 - Filed: 2022-09-08 - Published: 2023-01-05 - Granted: 2024-10-08 - Earliest claimed: 2019-05-06 - Assignees: Google Inc. - Applicants: Google LLC - Inventors: Asaf Aharoni; Eyal Segalis; Yaniv Leviathan - Web page: https://patentgrep.com/patent/US12112755B2 ## CPC - G10L 15/222: {Barge in, i.e. overridable guidance for interrupting prompts} - H04M 3/5158: {in combination with automated outdialling systems (devices for signalling identity of wanted subscriber H04M1/26)} - H04M 3/527: Centralised call answering arrangements not requiring operator intervention - G10L 25/78: Detection of presence or absence of voice signals (switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10) - G10L 2015/223: {Execution procedure of a spoken command} - H04M 3/4936: {Speech interaction details (speech recognition per se G10L15/00)} - H04M 2201/16: Sequence circuits - H04M 2201/39: using speech synthesis - H04M 2201/40: using speech recognition - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08) ## Abstract Methods, systems, and apparatus for an automated calling system are disclosed. Some implementations are directed to using a bot to initiate telephone calls and conduct telephone conversations with a user. The bot may be interrupted while providing synthesized speech during the telephone call. The interruption can be classified into one of multiple disparate interruption types, and the bot can react to the interruption based on the interruption type. Some implementations are directed to determining that a first user is placed on hold by a second user during a telephone conversation, and maintaining the telephone call in an active state in response to determining the first user hung up the telephone call. The first user can be notified when the second user rejoins the call, and a bot associated with the first user can notify the first user that the second user has rejoined the telephone call. ## Claims ### Claim 1 A method implemented by one or more processors, the method comprising: determining that a first user and a second user are engaged in a telephone call, wherein the first user is associated with a corresponding first computing device, and wherein the second user is associated with a corresponding second computing device; determining that the second user has placed the first user on hold; determining that the first user has hung up the corresponding first computing device; in response to determining that the first user has hung up the corresponding first phone and while the first user is on hold: causing a bot, that is associated with the corresponding first computing device of the first user, to maintain the telephone call in an active state; and determining whether the second user has rejoined the telephone call; and in response to determining that the second user has rejoined the telephone call: causing the bot, that is associated with the corresponding first computing device of the first user, to continue the telephone call with the second user, wherein causing the bot associated with the corresponding first computing device of the first user to continue the telephone call with the second user comprises: causing the bot to generate synthesized speech of the bot that conveys one or more details with respect to the first user initiating the telephone call; and causing the bot to provide, for output at the corresponding second computing device of the second user, the synthesized speech of the bot; causing the bot to provide, for output at the corresponding first computing device, a notification for the first user to rejoin the telephone call; determining that the first user has rejoined the telephone call; and in response to determining that the first user has rejoined the telephone call: causing the bot to cease the telephone conversation with the second user. ### Claim 2 (depends on 1) The method of claim 1, wherein causing the bot associated with the corresponding first computing device of the first user to continue the telephone call with the second user further comprises: causing the bot to generate additional synthesized speech of the bot that indicates the first user is not an active participant on the telephone call; and causing the bot to provide, for output at the corresponding second computing device of the second user, the additional synthesized speech of the bot. ### Claim 3 (depends on 1) The method of claim 1, further comprising: in response to determining that the first user has not rejoined the telephone call: causing the bot associated with the corresponding first computing device of the first user to place the second user on hold. ### Claim 4 (depends on 3) The method of claim 3, further comprising: in response to determining that the first user has not rejoined the telephone call within a threshold duration of time of causing the bot associated with the corresponding first computing device of the first user to place the second user on hold: causing the bot to terminate the telephone call. ### Claim 5 (depends on 1) The method of claim 1, further comprising: prior to the first user rejoining the telephone call, receiving, from the second user, a user utterance; causing the bot to generate a transcription of the user utterance; and causing the bot to include the notification for the first user to rejoin the telephone call in the transcription of the user utterance. ### Claim 6 (depends on 1) The method of claim 1, wherein maintaining the telephone call in the active state comprises: muting one or more microphones of the corresponding first computing device; and maintaining a telephonic connection between the corresponding first computing device and the corresponding second computing device. ### Claim 7 (depends on 6) The method of claim 6, in response to determining that the first user has rejoined the telephone call, further comprising: unmuting the one or more microphones of the corresponding first computing device. ### Claim 8 (depends on 1) The method of claim 1, wherein the bot is implemented at a remote server that is remote from the first computing device. ### Claim 9 (depends on 1) The method of claim 1, wherein the bot is implemented locally at the first computing device. ### Claim 10 A system comprising: one or more processors; and memory storing instructions that, when executed by the one or more processors, cause the one or more processors to: determine that a first user and a second user are engaged in a telephone call, wherein the first user is associated with a corresponding first computing device, and wherein the second user is associated with a corresponding second computing device; determine that the second user has placed the first user on hold; determine that the first user has hung up the corresponding first computing device; and in response to determining that the first user has hung up the corresponding first phone and while the first user is on hold: cause a bot, that is associated with the corresponding first computing device of the first user, to maintain the telephone call in an active state; and determine whether the second user has rejoined the telephone call; and in response to determining that the second user has rejoined the telephone call: cause the bot, that is associated with the corresponding first computing device of the first user, to continue the telephone call with the second user, wherein the instructions to cause the bot associated with the corresponding first computing device of the first user to continue the telephone call with the second user comprise instructions to: cause the bot to generate synthesized speech of the bot that conveys one or more details with respect to the first user initiating the telephone call; and cause the bot to provide, for output at the corresponding second computing device of the second user, the synthesized speech of the bot; cause the bot to provide, for output at the corresponding first computing device, a notification for the first user to rejoin the telephone call; determine that the first user has rejoined the telephone call; and in response to determining that the first user has rejoined the telephone call: cause the bot to cease the telephone conversation with the second user. ### Claim 11 (depends on 10) The system of claim 10, wherein the instructions to cause the bot associated with the corresponding first computing device of the first user to continue the telephone call with the second user comprise further instructions to: cause the bot to generate additional synthesized speech of the bot that indicates the first user is not an active participant on the telephone call; and cause the bot to provide, for output at the corresponding second computing device of the second user, the additional synthesized speech of the bot. ### Claim 12 (depends on 10) The system of claim 10, wherein the instructions further cause the one or more processors to: in response to determining that the first user has not rejoined the telephone call: cause the bot associated with the corresponding first computing device of the first user to place the second user on hold. ### Claim 13 (depends on 12) The system of claim 12, wherein the instructions further cause the one or more processors to: in response to determining that the first user has not rejoined the telephone call within a threshold duration of time of causing the bot associated with the corresponding first computing device of the first user to place the second user on hold: cause the bot to terminate the telephone call. ### Claim 14 (depends on 10) The system of claim 10, wherein the instructions further cause the one or more processors to: prior to the first user rejoining the telephone call, receive, from the second user, a user utterance; cause the bot to generate a transcription of the user utterance; and cause the bot to include the notification for the first user to rejoin the telephone call in the transcription of the user utterance. ### Claim 15 (depends on 10) The system of claim 10, wherein the instructions to maintain the telephone call in the active state comprise instructions to: mute one or more microphones of the corresponding first computing device; and maintain a telephonic connection between the corresponding first computing device and the corresponding second computing device. ### Claim 16 (depends on 15) The system of claim 15, in response to determining that the first user has rejoined the telephone call, wherein the instructions further cause the one or more processors to: unmute the one or more microphones of the corresponding first computing device. ### Claim 17 A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations, the operations comprising: determining that a first user and a second user are engaged in a telephone call, wherein the first user is associated with a corresponding first computing device, and wherein the second user is associated with a corresponding second computing device; determining that the second user has placed the first user on hold; determining that the first user has hung up the corresponding first computing device; in response to determining that the first user has hung up the corresponding first phone and while the first user is on hold: causing a bot, that is associated with the corresponding first computing device of the first user, to maintain the telephone call in an active state; and determining whether the second user has rejoined the telephone call; and in response to determining that the second user has rejoined the telephone call: causing the bot, that is associated with the corresponding first computing device of the first user, to continue the telephone call with the second user, wherein causing the bot associated with the corresponding first computing device of the first user to continue the telephone call with the second user comprises: causing the bot to generate synthesized speech of the bot that conveys one or more details with respect to the first user initiating the telephone call; and causing the bot to provide, for output at the corresponding second computing device of the second user, the synthesized speech of the bot; causing the bot to provide, for output at the corresponding first computing device, a notification for the first user to rejoin the telephone call; determining that the first user has rejoined the telephone call; and in response to determining that the first user has rejoined the telephone call: causing the bot to cease the telephone conversation with the second user.