# US 2019/0057698 A1 · In-call virtual assistant > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0057698 A1 - Document ID: US20190057698A1 - Status: Published application - Kind: A1 - Application number: 16/165,566 - Filed: 2018-10-19 - Published: 2019-02-21 - Earliest claimed: 2015-06-01 - Assignees: AffectLayer, Inc. - Applicants: AffectLayer, Inc. - Inventors: Roy Raanani; Russell Levy; Micha Yochanan Breakstone - Web page: https://patentgrep.com/patent/US20190057698A1 ## CPC - G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue - G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback} - G10L 15/02: Feature extraction for speech recognition; Selection of recognition unit - H04M 3/22: Arrangements for supervision, monitoring or testing - H04M 3/2281: {Call monitoring, e.g. for law enforcement purposes; Call tracing; Detection or prevention of malicious calls} - G10L 2015/221: {Announcement of recognition results} - G10L 2015/223: {Execution procedure of a spoken command} - H04M 2201/40: using speech recognition - H04M 2203/301: Management of recordings - H04M 2203/303: Marking - H04M 2203/552: Call annotations - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08) ## Abstract An in-call virtual assistant system monitors a real-time call, e.g., a call that is in progress, between multiple speakers, identifies a trigger and executes a specified task in response to the trigger. The virtual assistant system can be invoked by an explicit trigger or an implicit trigger. For example, an explicit trigger can be a voice command from one of the speakers in the call, such as “Ok Chorus, summarize the call” for summarizing the call. An implicit trigger can be an event that occurred in the call, or outside of the call but that is relevant to a speaker. For example, an event such as a speaker dropping off the call suddenly can be an implicit trigger that invokes the virtual assistant system to perform an associated task, such as notifying the remaining speakers on the call that one of the speakers dropped. ## Claims ### Claim 1 A computer-implemented method, comprising: receiving real-time call data of a call that is in progress between a representative and a customer; extracting multiple features from the real-time call data, wherein the multiple features include any of characteristics of the representative, characteristics of the customer and characteristics associated with a conversation of the call; monitoring the real-time call data to identify a trigger for invoking a virtual assistant component, wherein the virtual assistant component executes a specified task associated with the call; receiving the trigger to invoke the virtual assistant component; and executing, by the virtual assistant component and using the multiple features, the specified task. ### Claim 2 (depends on 1) The computer-implemented method of claim 1 further comprising: generating a result of the specified task, and generating a notification regarding the result of the specified task for at least one of the representative and customer. ### Claim 3 (depends on 2) The computer-implemented method of claim 2, wherein generating the notification includes generating at least one of a textual notification or an audio notification. ### Claim 4 (depends on 1) The computer-implemented method of claim 1, wherein receiving the trigger includes receiving a voice command from the representative for invoking the virtual assistant component. ### Claim 5 (depends on 4) The computer-implemented method of claim 4, wherein receiving the voice command includes receiving a specified word or a specified phrase from the representative that is explicitly indicative of invoking the virtual assistant component. ### Claim 6 (depends on 4) The computer-implemented method of claim 4, wherein receiving the voice command includes receiving the specified task from the representative. ### Claim 7 (depends on 1) The computer-implemented method of claim 1, wherein receiving the trigger includes determining an event associated with the call that is implicitly indicative of invoking the virtual assistant component. ### Claim 8 (depends on 7) The computer-implemented method of claim 7, wherein determining the event includes determining a first feature of the multiple features that is implicitly indicative of invoking the virtual assistant component. ### Claim 9 (depends on 8) The computer-implemented method of claim 8, wherein determining the first feature includes determining a set of words or phrases in the call that implicitly invokes the virtual assistant component. ### Claim 10 (depends on 8) The computer-implemented method of claim 8, wherein determining the first feature includes determining external data associated with the call that is implicitly indicative of invoking the virtual assistant component. ### Claim 11 (depends on 10) The computer-implemented method of claim 10, wherein determining the external data includes determining information associated with the customer based on metadata of the call. ### Claim 12 (depends on 10) The computer-implemented method of claim 10, wherein determining the external data includes determining information associated with the representative based on metadata of the call. ### Claim 13 (depends on 10) The computer-implemented method of claim 10, wherein determining the external data includes determining information associated with the conversation based on metadata of the call. ### Claim 14 (depends on 10) The computer-implemented method of claim 10, wherein determining the external data includes obtaining the external data from one or more computer servers accessible by the virtual assistant component using the metadata of the call. ### Claim 15 (depends on 1) The computer-implemented method of claim 1, wherein executing the specified task includes generating a summary of the conversation based on the multiple features. ### Claim 16 (depends on 15) The computer-implemented method of claim 15, wherein generating the summary includes generating the summary by performing a semantic analysis of the multiple features, the multiple features including one or more words or phrases in the call. ### Claim 17 (depends on 15) The computer-implemented method of claim 15, wherein generating the summary includes generating the summary using any of a rule-based, artificial intelligence (Al), machine-learning (ML), or natural language processing (NLP) techniques. ### Claim 18 (depends on 1) The computer-implemented method of claim 1, wherein executing the specified task includes generating an action item of the call based on the multiple features, the multiple features including one or more words or phrases in the call that are indicative of an action item. ### Claim 19 (depends on 1) The computer-implemented method of claim 1, wherein extracting the features includes extracting the features using any of an Al, NL, or NLP techniques. ### Claim 20 (depends on 1) The computer-implemented method of claim 1, wherein the call is at least one of a video call, an online meeting, a virtual reality-based call, an augmented reality-based call, or an email interaction between the customer and the representative. ### Claim 21 (depends on 1) The computer-implemented method of claim 1, wherein receiving the trigger includes receiving a series of voice commands from the representative, wherein a second command of the series of voice commands is received after the completion of a task of a first command of the series of voice commands by the virtual assistant component. ### Claim 22 A computer-readable storage medium storing computer-readable instructions, comprising: instructions for receiving real-time call data of a call that is in progress between multiple speakers; instructions for extracting multiple features from the real-time call data, wherein the multiple features include at least one of characteristics of a first speaker of the speakers and characteristics associated with a conversation of the call; instructions for receiving, by the virtual assistant component, a voice command for executing a specified task associated with the call; and instructions for executing, by the virtual assistant component and using the multiple features, the specified task. ### Claim 23 (depends on 22) The computer-readable storage medium of claim 22, wherein the instructions for executing the specified task include: instructions for analyzing the multiple features to obtain data for performing the specified task, and instructions for executing the specified task based on the data. ### Claim 24 (depends on 23) The computer-readable storage medium of claim 23, wherein the instructions for analyzing the multiple features include: instructions for obtaining the data from analyzing the multiple features, deriving the data from metadata of the call, or deriving the data based on data obtained from analyzing the multiple features. ### Claim 25 (depends on 22) The computer-readable storage medium of claim 22, wherein the instructions for receiving the voice command include: instructions for receiving one or more words from the first speaker for invoking the virtual assistant component. ### Claim 26 (depends on 25) The computer-readable storage medium of claim 25, wherein the instructions for receiving the voice command include: instructions for receiving a set of words from the first speaker that is indicative of the specified task. ### Claim 27 (depends on 22) The computer-readable storage medium of claim 22 further comprising: instructions for determining an event associated with the call that is implicitly indicative of invoking the virtual assistant component. ### Claim 28 (depends on 27) The computer-readable storage medium of claim 27, wherein the instructions for determining the event include: instructions for determining a first feature of the multiple features that is implicitly indicative of invoking the virtual assistant component, the first feature including one or more words from the call that implicitly invokes the virtual assistant component. ### Claim 29 (depends on 27) The computer-readable storage medium of claim 27, wherein the instructions for determining the event include: instructions for determining a first feature of the multiple features that is indicative of an event external to the call but associated with the first speaker that is implicitly indicative of invoking the virtual assistant component. ### Claim 30 (depends on 29) The computer-readable storage medium of claim 29, wherein the instructions for determining the first feature further include: instructions for analyzing, by the virtual assistant component, the first feature to determine the specified task, and instructions for executing the specified task by the virtual assistant component. ### Claim 31 A system, comprising: a processor; a first component that is configured to receive real-time call data of a call that is in progress between a representative and a customer; a second component that is configured to extract multiple features from the real-time call data, wherein the multiple features include any of characteristics of the representative, characteristics of the customer and characteristics associated with a conversation of the call; and a third component that is configured to monitor the real-time call data for a trigger to invoke a virtual assistant for executing a specified task, and to invoke the virtual assistant; and a fourth component that is configured to execute, using the multiple features, the specified task.