# US 12,118,986 B2 · System and method for automated processing of natural language using deep learning model encoding > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,118,986 B2 - Document ID: US12118986B2 - Status: Granted - Kind: B2 - Application number: 17/380,425 - Filed: 2021-07-20 - Published: 2023-01-26 - Granted: 2024-10-15 - Earliest claimed: 2021-07-20 - Assignees: Conduent Business Services, LLC - Applicants: Conduent Business Services, LLC - Inventors: Rijul Magu - Web page: https://patentgrep.com/patent/US12118986B2 ## CPC - G06F 40/30: Semantic analysis - G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning} - G06N 3/08: Learning methods - G10L 15/04: Segmentation; Word boundary detection - G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue - H04M 3/5175: {Call or contact centers supervision arrangements} - G06N 20/00: Machine learning - G10L 15/26: Speech to text systems (G10L15/08 takes precedence) - H04M 3/42221: {Conversation recording systems (at the subscriber's set H04M1/656)} - H04M 2201/40: using speech recognition - H04M 2203/303: Marking - H04M 2203/401: Performance feedback - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08) ## Abstract Automated systems and methods are provided for processing natural language, comprising obtaining first and second digitally-encoded speech representations, respectively corresponding to an agent script for and a voice recording of a telecommunication interaction; generating a similarity structure based on the speech representations, the similarity structure representing a degree of semantic similarity between the speech representations; matching markers in the first speech representation to markers in the second speech representation based on the similarity structure; and dividing the telecommunication interaction into a plurality of sections based on the matching. ## Claims ### Claim 1 A computer-implemented method for processing natural language, comprising: obtaining a first digitally-encoded speech representation corresponding to an agent script for a telecommunication interaction; obtaining a second digitally-encoded speech representation corresponding to a voice recording of the telecommunication interaction; generating a similarity structure based on the first digitally-encoded speech representation and the second digitally-encoded speech representation, the similarity structure representing a degree of semantic similarity between the first digitally-encoded speech representation and the second digitally-encoded speech representation; matching at least one first marker in the first digitally-encoded speech representation to at least one second marker in the second digitally-encoded speech representation based on the similarity structure, wherein matching further comprises identifying discrepancies between an order of markers of first digitally-encoded speech representation and an order of markers of second digitally-encoded speech representation; and dividing the telecommunication interaction into a plurality of sections based on the matching. ### Claim 2 (depends on 1) The method according to claim 1, wherein obtaining the first digitally-encoded speech representation includes applying a machine learning algorithm to the agent script to generate the first digitally-encoded speech representation, wherein the first digitally-encoded speech representation is a first numeric vector having a predetermined size. ### Claim 3 (depends on 2) The method according to claim 2, wherein obtaining the second digitally-encoded speech representation includes: generating a transcript from the voice recording; and applying the machine learning algorithm to the transcript to generate the second digitally-encoded speech representation, wherein the first digitally-encoded speech representation is a second numeric vector having the predetermined size. ### Claim 4 (depends on 3) The method according to claim 3, wherein generating the similarity structure includes applying a vector operation to the first numeric vector and the second numeric vector, thereby to generate a similarity matrix including a plurality of matrix elements. ### Claim 5 (depends on 4) The method according to claim 4, wherein matching the at least one first marker to the at least one second marker includes: comparing the plurality of matrix elements to a predetermined similarity threshold value; and discarding respective ones of the plurality of matrix elements which are less than the similarity threshold value. ### Claim 6 (depends on 5) The method according to claim 5, wherein dividing the telecommunication interaction into the plurality of sections includes: applying a clustering algorithm to each of the plurality of matrix elements which have not been discarded; and generating at least one section start timestamp and at least one section stop timestamp. ### Claim 7 (depends on 1) The method according to claim 1, wherein the first digitally-encoded speech representation includes a first plurality of embeddings respectively corresponding to a plurality of expected utterances in the telecommunication interaction, and the second digitally-encoded speech representation includes a second plurality of embeddings respectively corresponding to a plurality of actual utterances in the telecommunication interaction. ### Claim 8 (depends on 7) The method according to claim 7, wherein identifying discrepancies between the order of markers of the first digitally-encoded speech representation and the order of markers of the second digitally-encoded speech representation further comprises: identifying which of the plurality of expected utterances do not have a corresponding one of the plurality of actual utterances; and identifying which of the plurality of expected utterances have multiple corresponding ones of the plurality of actual utterances. ### Claim 9 (depends on 7) The method according to claim 7, wherein the order of markers of first digitally-encoded speech representation comprises an expected order of the plurality of expected utterances, the order of markers of second digitally-encoded speech representation comprises an actual order of the plurality of actual utterances, and the method further comprises identifying discrepancies between the expected order and the actual order. ### Claim 10 (depends on 1) The method according to claim 1, further comprising calculating a duration of at least one of the plurality of sections. ### Claim 11 A computing system for processing natural language, the system comprising: at least one electronic processor; and a non-transitory computer-readable medium storing instructions that, when executed by the at least one electronic processor, cause the at least one electronic processor to perform operations comprising: obtaining a first digitally-encoded speech representation corresponding to an agent script for a telecommunication interaction, obtaining a second digitally-encoded speech representation corresponding to a voice recording of the telecommunication interaction, generating a similarity structure based on the first digitally-encoded speech representation and the second digitally-encoded speech representation, the similarity structure representing a degree of semantic similarity between the first digitally-encoded speech representation and the second digitally-encoded speech representation, matching at least one first marker in the first digitally-encoded speech representation to at least one second marker in the second digitally-encoded speech representation based on the similarity structure, wherein matching further comprises identifying discrepancies between an order of markers of first digitally-encoded speech representation and an order of markers of second digitally-encoded speech representation, and dividing the telecommunication interaction into a plurality of sections based on the matching. ### Claim 12 (depends on 11) The system according to claim 11, wherein obtaining the first digitally-encoded speech representation includes applying a machine learning algorithm to the agent script to generate the first digitally-encoded speech representation, wherein the first digitally-encoded speech representation is a first numeric vector having a predetermined size. ### Claim 13 (depends on 12) The system according to claim 12, wherein obtaining the second digitally-encoded speech representation includes: generating a transcript from the voice recording; and applying the machine learning algorithm to the transcript to generate the second digitally-encoded speech representation, wherein the first digitally-encoded speech representation is a second numeric vector having the predetermined size. ### Claim 14 (depends on 13) The system according to claim 13, wherein generating the similarity structure includes applying a vector operation to the first numeric vector and the second numeric vector, thereby to generate a similarity matrix including a plurality of matrix elements. ### Claim 15 (depends on 14) The system according to claim 14, wherein matching the at least one first marker to the at least one second marker includes: comparing the plurality of matrix elements to a predetermined similarity threshold value; and discarding respective ones of the plurality matrix elements which are less than the similarity threshold value. ### Claim 16 (depends on 15) The system according to claim 15, wherein dividing the telecommunication interaction into the plurality of sections includes: applying a clustering algorithm to each of the plurality of matrix elements which have not been discarded; and generating at least one section start timestamp and at least one section stop timestamp. ### Claim 17 (depends on 11) The system according to claim 11, wherein obtaining the first digitally-encoded speech representation includes retrieving a predetermined numeric vector from a memory of the system, the numeric vector including a plurality of elements respectively corresponding to subsections of the agent script. ### Claim 18 (depends on 11) The system according to claim 11, wherein the first digitally- encoded speech representation includes a first plurality of embeddings respectively corresponding to a plurality of expected utterances in the telecommunication interaction, and the second digitally-encoded speech representation includes a second plurality of embeddings respectively corresponding to a plurality of actual utterances in the telecommunication interaction. ### Claim 19 (depends on 18) The system according to claim 18, wherein identifying discrepancies between the order of markers of the first digitally-encoded speech representation and the order of markers of the second digitally-encoded speech representation further comprises: identifying which of the plurality of expected utterances do not have a corresponding one of the plurality of actual utterances; and identifying which of the plurality of expected utterances have multiple corresponding ones of the plurality of actual utterances. ### Claim 20 (depends on 11) The system according to claim 11, the operations further comprising calculating a duration of at least one of the plurality of sections.