# US 12,010,268 B2 · Partial automation of text chat conversations > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,010,268 B2 - Document ID: US12010268B2 - Status: Granted - Kind: B2 - Application number: 17/574,047 - Filed: 2022-01-12 - Published: 2022-05-05 - Granted: 2024-06-11 - Earliest claimed: 2018-10-02 - Assignees: Verint Americas Inc. - Applicants: Verint Americas Inc. - Inventors: Ian Beaver - Web page: https://patentgrep.com/patent/US12010268B2 ## CPC - G06F 16/35: Clustering; Classification - H04M 3/5166: {in combination with interactive voice response systems or voice portals, e.g. as front-ends} - G06F 16/3329: Natural language query formulation - G06F 16/36: Creation of semantic tools, e.g. ontology or thesauri - G06F 40/30: Semantic analysis - G06F 40/35: Discourse or dialogue representation - H04L 51/02: using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages - H04M 3/5183: {Call or contact centers with computer-telephony arrangements} - H04M 2203/404: Collaboration among agents - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08) - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract To allow the human customer service agents to specialize in the instances where human service is preferred, but to scale to the volume of large call centers, systems and methods are provided in which human agents and intelligent virtual assistants (IVAs) co-handle a conversation with a customer. IVAs handle simple or moderate tasks, and human agents are used for those tasks that require or would benefit from human compassion or special handling. Instead of starting the conversation with an IVA and then escalating or passing control of the conversation to a human to complete, the IVAs and human agents work together on a conversation. ## Claims ### Claim 1 A method for partially automating a text chat conversation, the method comprising: selecting a plurality of samples from a plurality of unlabeled samples, wherein the unlabeled samples comprise pairs of text strings comprising a user input and a corresponding live agent response from a chat conversation, the selected plurality of samples comprising such pairs for which a language model employed by an intelligent virtual assistant (IVA) has not been previously trained; receiving a labeled sample corresponding to each sample of the plurality of samples; adding each labeled sample to a labeled training set; training the language model with the labeled training set; and using the language model in a text conversation between a user computing device and the IVA. ### Claim 2 (depends on 1) The method of claim 1, wherein the language model comprises a plurality of machine learning models. ### Claim 3 (depends on 1) The method of claim 1, further comprising receiving the plurality of unlabeled samples from a stream of unlabeled samples. ### Claim 4 (depends on 3) The method of claim 3, wherein the stream of unlabeled samples is received from a plurality of IVAs. ### Claim 5 (depends on 1) The method of claim 1, wherein selecting the plurality of samples is performed by the language model choosing the unlabeled samples that would be most informative to know the label for. ### Claim 6 (depends on 1) The method of claim 1, wherein receiving the labeled sample comprises receiving a label for each sample of the plurality of samples from an oracle. ### Claim 7 (depends on 1) The method of claim 1, further comprising monitoring the text conversation between the user computing device and the IVA and adjusting the labeled training set based on the monitoring. ### Claim 8 (depends on 7) The method of claim 7, wherein the monitoring is performed continuously during the text conversation. ### Claim 9 A method for partially automating a text chat conversation, the method comprising: collecting a plurality of input-response pairs from a plurality of intelligent virtual assistants (IVAs), wherein the plurality of input-response pairs comprise pairs of text strings comprising a user input and a corresponding live agent response from a chat conversation; receiving a label for each of the plurality of input-response pairs; and training a language model with the labeled input-response pairs. ### Claim 10 (depends on 9) The method of claim 9, further comprising adding each labeled input-response pair to a labeled training set, wherein training the language model uses the labeled training set. ### Claim 11 (depends on 9) The method of claim 9, wherein each input-response pair is from a text chat conversation of at least one of the plurality of IVAs. ### Claim 12 (depends on 9) The method of claim 9, further comprising using the language model in a text conversation. ### Claim 13 (depends on 12) The method of claim 12, further comprising continuously monitoring the text conversation and adjusting a labeled training set based on the monitoring. ### Claim 14 A system for partially automating a text chat conversation, the system comprising: an agent computing device with human agent text conversation capability and intelligent virtual assistant (IVA) text conversation ability; an active learning engine configured to: select a plurality of samples from a plurality of unlabeled samples, wherein the unlabeled samples comprise pairs of text strings comprising a user input and a corresponding live agent response from a chat conversation, the selected plurality of samples comprising such pairs for which a language model employed by the IVA has not been previously trained; receive a labeled sample corresponding to each sample of the plurality of samples; add each labeled sample to a labeled training set; and train the language model with the labeled training set; and a natural language understanding (NLU) component configured to use the language model in a text conversation between a user computing device and an IVA. ### Claim 15 (depends on 14) The system of claim 14, wherein the language model comprises a plurality of machine learning models. ### Claim 16 (depends on 14) The system of claim 14, wherein the active learning engine is further configured to receive the plurality of unlabeled samples from a stream of unlabeled samples. ### Claim 17 (depends on 16) The system of claim 16, wherein the stream of unlabeled samples is received from a plurality of IVAs. ### Claim 18 (depends on 14) The system of claim 14, wherein the NLU component is further configured to monitor the text conversation between the user computing device and IVA, and the active learning engine is further configured to adjust the labeled training set based on the monitoring. ### Claim 19 (depends on 18) The system of claim 18, wherein the monitoring is performed continuously during the text conversation. ### Claim 20 (depends on 14) The system of claim 14, wherein the NLU component comprises an intent determination engine configured to determine an intent of the text conversation from a plurality of intents.