# US 11,308,944 B2 · Intent boundary segmentation for multi-intent utterances > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,308,944 B2 - Document ID: US11308944B2 - Status: Granted - Kind: B2 - Application number: 16/816,600 - Filed: 2020-03-12 - Published: 2021-09-16 - Granted: 2022-04-19 - Earliest claimed: 2020-03-12 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: Ming Tan; Haoyu Wang; Saloni Potdar; Yang Yu; Navneet N. Rao; Haode Qi - Web page: https://patentgrep.com/patent/US11308944B2 ## CPC - G06F 40/30: Semantic analysis - G10L 15/183: using context dependencies, e.g. language models - G06F 16/90332: {Natural language query formulation or dialogue systems} - G06N 20/00: Machine learning - G10L 15/063: {Training} - G10L 15/1822: {Parsing for meaning understanding} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract A mechanism is provided for implementing an intent segmentation mechanism that segments intent boundaries for multi-intent utterances in a conversational agent. For each term of a set of terms in the utterance from a real-time chat session, a set of adversarial utterances is generated for the utterance. An influence of changing each term is determined so as to identify a term importance value. Utilizing the term importance value, one or more of a change in ranking of the intent of the utterance or a change in confidence with regard to the intent of the utterance is identified. An entropy-based segmentation of the utterance into a plurality of candidate partitions is performed. An associated intent and entropy value are then assigned. Based on a segment with minimum entropy, a call associated with the real-time chat session is directed to an operation associated with an intent of the segment with minimum entropy. ## Claims ### Claim 1 A method, in a data processing system comprising at least one processor and at least one memory, wherein the at least one memory comprises instructions that are executed by the at least one processor to configure the at least one processor to implement an intent segmentation mechanism that segments intent boundaries for multi-intent utterances in a conversational agent, the method comprising: responsive to identifying one or more possible intents in an utterance from real-time chat information from a real-time chat session, for each term of a set of terms in the utterance, generating a set of adversarial utterances for the utterance; determining an influence of changing each term by: determining a meaning of the utterance without the alternative term; determining a meaning of the utterance with the alternative term; and comparing the meaning of the utterance without the alternative term to the meaning of the utterance with the alternative term so as identify a term importance value; utilizing the term importance value of each term in the set of terms in the utterance, identifying one or more of a change in ranking of the intent of the utterance or a change in confidence with regard to the intent of the utterance; utilizing the term importance value for each term in the set of terms in the utterance, performing entropy-based segmentation of the utterance into a plurality of candidate partitions; assigning an associated intent based on ranking and confidence and entropy value to each segment of the utterance based on a sum of term importance values of the terms in the respective segment; and based on a segment with minimum entropy, directing a call associated with the real-time chat session to an operation associated with the associated intent of the segment with minimum entropy. ### Claim 2 (depends on 1) The method of claim 1, further comprising: responsive to completing a portion of the call associated with the intent of the segment with minimum entropy, directing the call associated with the real-time chat session to an further operation associated with an intent of the segment with next minimum entropy. ### Claim 3 (depends on 1) The method of claim 1, further comprising: utilizing the identified segmentation along with the identified intent as additional training data for further training of the conversational agent. ### Claim 4 (depends on 1) The method of claim 1, wherein the one or more possible intents in the utterance from the real-time chat information are identified by the method comprising: identifying one or more entities within the utterance using natural language processing; and identifying a possible intent associated with each identified entity in the one or more entities utilizing one or more of semantic classification or text classification. ### Claim 5 (depends on 1) The method of claim 1, wherein generating the set of adversarial utterances for the utterance performs one or more of replacing terms with paraphrasing terms, randomly sampling replacement words by language models, or utilizing replacement terms that involve typos. ### Claim 6 (depends on 1) The method of claim 1, wherein generating the set of adversarial utterances for the utterance deletes a word from the utterance. ### Claim 7 (depends on 1) The method of claim 1, wherein determining the influence of changing each term utilizes the following equations: ΔL=Sbefore−Safter S = ∝ 1 log ⁡ ( p ⁢ o ⁢ s + 1 ) + β · confidence where is a result of a difference of a score before (Sbefore) the change of the term and a score after (Safter) the change of the term and the evaluation score S considers the effects of both position (pos) in the meaning prediction list for this meaning and the probability (confidence) of the meaning. ### Claim 8 A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a data processing system, causes the data processing system to implement an abnormal entity detection mechanism that facilitates detecting abnormal entities in real-time response systems through weak supervision, and further causes the data processing system to: responsive to identifying one or more possible intents in an utterance from real-time chat information from a real-time chat session, for each term of a set of terms in the utterance, generate a set of adversarial utterances for the utterance; determine an influence of changing each term by: determining a meaning of the utterance without the alternative term; determining a meaning of the utterance with the alternative term; and comparing the meaning of the utterance without the alternative term to the meaning of the utterance with the alternative term so as identify a term importance value; utilizing the term importance value of each term in the set of terms in the utterance, identify one or more of a change in ranking of the intent of the utterance or a change in confidence with regard to the intent of the utterance; utilizing the term importance value for each term in the set of terms in the utterance, perform entropy-based segmentation of the utterance into a plurality of candidate partitions; assign an associated intent based on ranking and confidence and entropy value to each segment of the utterance based on a sum of term importance values of the terms in the respective segment; and based on a segment with minimum entropy, direct a call associated with the real-time chat session to an operation associated with the associated intent of the segment with minimum entropy. ### Claim 9 (depends on 8) The computer program product of claim 8, wherein the computer readable program further causes the data processing system to: responsive to completing a portion of the call associated with the intent of the segment with minimum entropy, direct the call associated with the real-time chat session to an further operation associated with an intent of the segment with next minimum entropy. ### Claim 10 (depends on 8) The computer program product of claim 8, wherein the computer readable program further causes the data processing system to: utilize the identified segmentation along with the identified intent as additional training data for further training of the conversational agent. ### Claim 11 (depends on 8) The computer program product of claim 8, wherein the one or more possible intents in the utterance from the real-time chat information are identified by the computer readable program further causing the data processing system to: identify one or more entities within the utterance using natural language processing; and identify a possible intent associated with each identified entity in the one or more entities utilizing one or more of semantic classification or text classification. ### Claim 12 (depends on 8) The computer program product of claim 8, wherein the computer readable program generates the set of adversarial utterances for the utterance by causing the data processing system to perform one or more of replacing terms with paraphrasing terms, randomly sampling replacement words by language models, or utilizing replacement terms that involve typos. ### Claim 13 (depends on 8) The computer program product of claim 8, wherein the computer readable program generates the set of adversarial utterances for the utterance by causing the data processing system to delete a word from the utterance. ### Claim 14 (depends on 8) The computer program product of claim 8, wherein the computer readable program determines the influence of changing each term by causing the data processing system to utilize the following equations: Δ ⁢ ⁢ L = S before - S after S = ∝ 1 log ⁡ ( p ⁢ o ⁢ s + 1 ) + β · confidence where is a result of a difference of a score before (Sbefore) the change of the term and a score after (Safter) the change of the term and the evaluation score S considers the effects of both position (pos) in the meaning prediction list for this meaning and the probability (confidence) of the meaning. ### Claim 15 An apparatus comprising: at least one processor; and at least one memory coupled to the at least one processor, wherein the at least one memory comprises instructions which, when executed by the at least one processor, cause the at least one processor to implement an abnormal entity detection mechanism that facilitates detecting abnormal entities in real-time response systems through weak supervision, and further cause the at least one processor to: responsive to identifying one or more possible intents in an utterance from real-time chat information from a real-time chat session, for each term of a set of terms in the utterance, generate a set of adversarial utterances for the utterance; determine an influence of changing each term by: determining a meaning of the utterance without the alternative term; determining a meaning of the utterance with the alternative term; and comparing the meaning of the utterance without the alternative term to the meaning of the utterance with the alternative term so as identify a term importance value; utilizing the term importance value of each term in the set of terms in the utterance, identify one or more of a change in ranking of the intent of the utterance or a change in confidence with regard to the intent of the utterance; utilizing the term importance value for each term in the set of terms in the utterance, perform entropy-based segmentation of the utterance into a plurality of candidate partitions; assign an associated intent based on ranking and confidence and entropy value to each segment of the utterance based on a sum of term importance values of the terms in the respective segment; and based on a segment with minimum entropy, direct a call associated with the real-time chat session to an operation associated with the associated intent of the segment with minimum entropy. ### Claim 16 (depends on 15) The apparatus of claim 15, wherein the instructions further cause the at least one processor to: responsive to completing a portion of the call associated with the intent of the segment with minimum entropy, direct the call associated with the real-time chat session to an further operation associated with an intent of the segment with next minimum entropy. ### Claim 17 (depends on 15) The apparatus of claim 15, wherein the instructions further cause the at least one processor to: utilize the identified segmentation along with the identified intent as additional training data for further training of the conversational agent. ### Claim 18 (depends on 15) The apparatus of claim 15, wherein the one or more possible intents in the utterance from the real-time chat information are identified by the instructions further causing the at least one processor to: identify one or more entities within the utterance using natural language processing; and identify a possible intent associated with each identified entity in the one or more entities utilizing one or more of semantic classification or text classification. ### Claim 19 (depends on 15) The apparatus of claim 15, wherein the instructions generate the set of adversarial utterances for the utterance by causing the at least one processor to perform one or more of replacing terms with paraphrasing terms, randomly sampling replacement words by language models, or utilizing replacement terms that involve typos. ### Claim 20 (depends on 15) The apparatus of claim 15, wherein the instructions generate the set of adversarial utterances for the utterance by causing the at least one processor to delete a word from the utterance.