# US 11,157,706 B2 · Omnichannel data communications system using artificial intelligence (AI) based machine learning and predictive analysis > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,157,706 B2 - Document ID: US11157706B2 - Status: Granted - Kind: B2 - Application number: 16/581,124 - Filed: 2019-09-24 - Published: 2020-09-17 - Granted: 2021-10-26 - Earliest claimed: 2019-03-15 - Assignees: Contactengine Limited - Applicants: ContactEngine Limited - Inventors: Andrew Mark Ayers; Giles Bryan; Mark Kingsley Smith - Web page: https://patentgrep.com/patent/US11157706B2 ## CPC - G06F 40/30: Semantic analysis - G06F 40/35: Discourse or dialogue representation - G06F 40/205: Parsing - G06F 40/263: Language identification - G06F 40/40: Processing or translation of natural language (natural language analysis G06F40/20; semantic analysis G06F40/30) - G06F 40/56: Natural language generation - G06N 3/04: Architecture, e.g. interconnection topology - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 5/045: Explanation of inference; Explainable artificial intelligence [XAI]; Interpretable artificial intelligence - G06N 20/00: Machine learning - G06N 20/20: Ensemble learning - H04L 51/02: using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages - G06N 3/045: Combinations of networks - G06N 5/022: Knowledge engineering; Knowledge acquisition - G06Q 30/016: After-sales - H04L 51/216: Handling conversation history, e.g. grouping of messages in sessions or threads - H04W 4/025: {using location based information parameters} - H04W 4/14: Short messaging services, e.g. short message services [SMS] or unstructured supplementary service data [USSD] - H04W 4/21: for social networking applications - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR - H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72) ## Abstract A system for determining context and intent in a conversation using machine learning (ML) based artificial intelligence (AI) in omnichannel data communications is disclosed. The system may comprise a data store to store and manage data within a network, a server to facilitate operations using information from the one or more data stores, and a ML-based AI subsystem to communicate with the server and the data store in the network. The ML-based AI subsystem may comprise a data access interface to receive data associated with a conversation with a user via a communication channel. The ML-based AI subsystem may comprise a processor to provide a proactive, adaptive, and intelligent conversation by applying hierarchical multi-intent data labeling framework, training at least one model with training data, and generating and deploying a production-ready model based on the trained and retained at least one model. ## Claims ### Claim 1 A system, comprising: a data store to store and to manage data within a network; a server to facilitate operations using information from the data store; a machine learning (ML)-based artificial intelligence (AI) subsystem to communicate with the server and the data store in the network, the ML-based AI subsystem comprising: a data access interface to: receive data associated with a conversation with a user via a communication channel, the data for training at least one model to determine intent in the conversation with the user; and a processor to: select any unlabeled data from the data associated with the conversation from the user, the selected unlabeled data having probability of impacting the at least one model; provide a hierarchical multi-intent data labeling framework for labeling a representative subset of data from at least one of the data associated with the conversation with the user or the selected unlabeled data; train the at least one model based on the labeled representative subset of data using a plurality of parameter choices; retain at least one candidate model set from the at least one trained models based on global metric performance; generate a production-ready model based on the retained at least one candidate model set, wherein generating the production-ready model comprises: applying a fine-tune intent level evaluation specific to any particular client workflow by providing at least one of the following: providing additional training data from auxiliary models to boost specific intent performance; or providing data oversampling with layer level fine-tuning techniques to correct or augment specific intent performance issues; and creating the production-ready model based on an ensemble of successful fine-tuned candidate models from the retained at least one candidate model set; and deploy the production-ready model for intent determination in the conversation with the user. ### Claim 2 (depends on 1) The system of claim 1, wherein the communication channel comprises at least one of telephone, email, simple message service (SMS), mobile device application, video conference, website, digital conversational entity, and social network platform. ### Claim 3 (depends on 1) The system of claim 1, wherein the data access interface further receives data from a data source, the data source comprising at least one of a website, a document, enterprise resource planning (ERP) system, a database, a web feed, a sensor, a geolocation data source, a server, an analytics tool, a mobile device, and a reporting system. ### Claim 4 (depends on 1) The system of claim 1, wherein the conversation is a multilingual conversation based on the data received from the user or information identifying the user. ### Claim 5 (depends on 1) The system of claim 1, further wherein cloud-distributed hyper-parameter tuning and ranges for the plurality of parameter choices are predetermined. ### Claim 6 (depends on 1) The system of claim 1, wherein the plurality of parameter choices is searched and selected randomly via a cloud computing cluster to test development sets to determine which of the at least one model to retain. ### Claim 7 (depends on 1) The system of claim 1, wherein global metric performance is based on at least one global numerical metric established for each client, wherein the at least one global numerical metric comprising at least one of precision and recall metrics of given intents or importance of identifying particular intents for the given client. ### Claim 8 (depends on 1) The system of claim 1, wherein deployment of the production-ready model is cloud-agnostic. ### Claim 9 A method for determining intent in a digital conversation, comprising: receiving, by a processor via a data access interface, data associated with a conversation with a user via a communication channel, the data for training at least one model to determine intent in the conversation with the user; selecting, by the processor, any unlabeled data from the data associated with the conversation from the user, the selected unlabeled data having probability of impacting the at least one model; providing, by the processor, a hierarchical multi-intent data labeling framework for labeling a representative subset of data from at least one of the data associated with the conversation with the user or the selected unlabeled data; training, by the processor, the at least one model based on the labeled representative subset of data using a plurality of parameter choices; retaining, by the processor, at least one candidate model set from the at least one trained models based on global metric performance; generating, by the processor, a production-ready model based on the retained at least one candidate model set, wherein generating the production-ready model comprises: applying a fine-tune intent level evaluation specific to any particular client workflow by providing at least one of the following: providing additional training data from auxiliary models to boost specific intent performance; or providing data oversampling with layer level fine-tuning techniques to correct or augment specific intent performance issues; and creating the production-ready model based on an ensemble of successful fine-tuned candidate models from the retained at least one candidate model set; and deploying, by the processor, the production-ready model for intent determination in the conversation with the user. ### Claim 10 (depends on 9) The method of claim 9, wherein the communication channel comprises at least one of telephone, email, simple message service (SMS), mobile device application, video conference, website, digital conversational entity, and social network platform. ### Claim 11 (depends on 9) The method of claim 9, wherein the data access interface further receives data from a data source, the data source comprising at least one of a website, a document, enterprise resource planning (ERP) system, a database, a web feed, a sensor, a geolocation data source, a server, an analytics tool, a mobile device, and a reporting system. ### Claim 12 (depends on 9) The method of claim 9, wherein the conversation is a multilingual conversation based on the data received from the user or information identifying the user. ### Claim 13 (depends on 9) The method of claim 9, further wherein cloud-distributed hyper-parameter tuning and ranges for the plurality of parameter choices are predetermined. ### Claim 14 (depends on 9) The method of claim 9, wherein the plurality of parameter choices is searched and selected randomly via a cloud computing cluster to test development sets to determine which of the at least one model to retain. ### Claim 15 (depends on 9) The method of claim 9, wherein global metric performance is based on at least one global numerical metric established for each client, wherein the at least one global numerical metric comprising at least one of precision and recall metrics of given intents or importance of identifying particular intents for the given client. ### Claim 16 (depends on 9) The method of claim 9, wherein deployment of the production-ready model is cloud-agnostic. ### Claim 17 A non-transitory computer-readable storage medium having an executable stored thereon, which when executed instructs a processor to perform the following: receiving data associated with a conversation with a user via a communication channel, the data for training at least one model to determine intent in the conversation with the user; selecting any unlabeled data from the data associated with the conversation from the user, the selected unlabeled data having probability of impacting the at least one model; providing a hierarchical multi-intent data labeling framework for labeling a representative subset of data from at least one of the data associated with the conversation with the user or the selected unlabeled data; training the at least one model based on the labeled representative subset of data using a plurality of parameter choices; retaining at least one candidate model set from the at least one trained models based on global metric performance; generating a production-ready model based on the retained at least one candidate model set, wherein generating the production-ready model comprises: applying a fine-tune intent level evaluation specific to any particular client workflow by providing at least one of the following: providing additional training data from auxiliary models to boost specific intent performance; or providing data oversampling with layer level fine-tuning techniques to correct or augment specific intent performance issues; and creating the production-ready model based on an ensemble of successful fine-tuned candidate models from the retained at least one candidate model set; and deploying the production-ready model for intent determination in the conversation with the user.