# US 10,771,627 B2 · Personalized support routing based on paralinguistic information > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,771,627 B2 - Document ID: US10771627B2 - Status: Granted - Kind: B2 - Application number: 16/826,931 - Filed: 2020-03-23 - Published: 2020-07-09 - Granted: 2020-09-08 - Earliest claimed: 2016-10-27 - Assignees: Intuit Inc. - Applicants: Intuit Inc. - Inventors: Benjamin Indyk; Igor A. Podgorny; Raymond Chan - Web page: https://patentgrep.com/patent/US10771627B2 ## CPC - H04M 3/5235: {Dependent on call type or called number [DNIS]} - G10L 15/02: Feature extraction for speech recognition; Selection of recognition unit - G10L 15/14: using statistical models, e.g. Hidden Markov Models [HMMs] (G10L15/18 takes precedence) - G10L 25/63: for estimating an emotional state - H04M 3/42068: {where the identifier is used to access a profile} - H04M 3/5233: {Operator skill based call distribution} - H04M 2201/40: using speech recognition - H04M 2203/2038: Call context notifications - H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08) - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract Embodiments presented herein provide techniques for inferring the current emotional state of a user based on paralinguistic features derived from audio input from that user. If the emotional state meets triggering conditions, the system provides the user with a prompt which allows the user to connect with a support agent. If the user accepts, the system selects a support agent for the user based on the predicted emotional state and on attributes of the support agent found in an agent profile. The system can also determine a priority level for the user based on the score and based on a profile of the user and determine where to place the user in a queue for the support agent. ## Claims ### Claim 1 A computer-implemented method, comprising: receiving a set of audio recordings and corresponding score labels measuring emotional states; extracting a set of paralinguistic features from each audio recording of the set of audio recordings; generating a set of training instances, wherein each training instance of the set of training instances is based on an audio recording of the set of audio recordings; identifying a subset of non-redundant paralinguistic features correlated to the score labels; training a predictive model, based on the subset of non-redundant paralinguistic features, to predict a target variable; and generating the trained predictive model. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein the training instance comprises: the set of paralinguistic features extracted from a corresponding audio recording, and the score label of the corresponding audio recording. ### Claim 3 (depends on 1) The computer-implemented method of claim 1, wherein the score label represents a value of the target variable that the predictive model is trained to predict. ### Claim 4 (depends on 1) The computer-implemented method of claim 1, wherein the target variable is a score measuring an emotional state. ### Claim 5 (depends on 1) The computer-implemented method of claim 1, wherein the identification of the subset of non-redundant paralinguistic features further comprises applying a feature-selection technique. ### Claim 6 (depends on 1) The computer-implemented method of claim 1, wherein the trained predictive model is stored in a format accessible for support routing. ### Claim 7 (depends on 1) The computer-implemented method of claim 1, wherein the predictive model is a neural network that iteratively adjusts weights within layers of network nodes based on an error term defined by a backpropagation technique. ### Claim 8 A system, comprising: a processor; and a memory storing instructions, which when executed by the processor perform a method comprising: receiving a set of audio recordings and corresponding score labels measuring emotional states; extracting a set of paralinguistic features from each audio recording of the set of audio recordings; generating a set of training instances, wherein each training instance of the set of training instances is based on an audio recording of the set of audio recordings; identifying a subset of non-redundant paralinguistic features correlated to the score labels; training a predictive model, based on the subset of non-redundant paralinguistic features, to predict a target variable; and generating the trained predictive model. ### Claim 9 (depends on 8) The system of claim 8, wherein the training instance comprises: the set of paralinguistic features extracted from a corresponding audio recording, and the score label of the corresponding audio recording. ### Claim 10 (depends on 8) The system of claim 8, wherein the score label represents a value of the target variable that the predictive model is trained to predict. ### Claim 11 (depends on 8) The system of claim 8, wherein the target variable is a score measuring an emotional state. ### Claim 12 (depends on 8) The system of claim 8, wherein the identification of the subset of non-redundant paralinguistic features further comprises applying a feature-selection technique. ### Claim 13 (depends on 8) The system of claim 8, wherein the trained predictive model is stored in a format accessible for support routing. ### Claim 14 (depends on 8) The system of claim 8, wherein the predictive model is a neural network that iteratively adjusts weights within layers of network nodes based on an error term defined by a backpropagation technique. ### Claim 15 A non-transitory computer readable storage medium comprising instructions, which when executed by a processor of a computing device, cause the computing device to perform a method for generating a predictive model, the method comprising: receiving a set of audio recordings and corresponding score labels measuring emotional states; extracting a set of paralinguistic features from each audio recording of the set of audio recordings; generating a set of training instances, wherein each training instance of the set of training instances is based on an audio recording of the set of audio recordings; identifying a subset of non-redundant paralinguistic features correlated to the score labels; training a predictive model, based on the subset of non-redundant paralinguistic features, to predict a target variable; and generating the trained predictive model. ### Claim 16 (depends on 15) The non-transitory computer readable storage medium of claim 15, wherein the training instance comprises: the set of paralinguistic features extracted from a corresponding audio recording, and the score label of the corresponding audio recording. ### Claim 17 (depends on 15) The non-transitory computer readable storage medium of claim 15, wherein the score label represents a value of the target variable that the predictive model is trained to predict. ### Claim 18 (depends on 15) The non-transitory computer readable storage medium of claim 15, wherein the target variable is a score measuring an emotional state. ### Claim 19 (depends on 15) The non-transitory computer readable storage medium of claim 15, wherein the identification of the subset of non-redundant paralinguistic features further comprises applying a feature-selection technique. ### Claim 20 (depends on 15) The non-transitory computer readable storage medium of claim 15, wherein the trained predictive model is stored in a format accessible for support routing.