# US 10,484,540 B2 · Systems and methods for customer sentiment prediction and depiction

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,484,540 B2
- Document ID: US10484540B2
- Status: Granted
- Kind: B2
- Application number: 15/997,745
- Filed: 2018-06-05
- Published: 2018-10-04
- Granted: 2019-11-19
- Earliest claimed: 2016-12-22
- Assignees: Capital One Services, LLC
- Applicants: Capital One Services, LLC
- Inventors: Xu Benjamin Fang; Lewis Wong
- Web page: https://patentgrep.com/patent/US10484540B2

## CPC

- H04M 3/5175: {Call or contact centers supervision arrangements}
- G06Q 30/016: After-sales
- G10L 21/10: Transforming into visible information
- H04L 51/02: using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages
- H04M 3/5166: {in combination with interactive voice response systems or voice portals, e.g. as front-ends}
- G10L 25/63: for estimating an emotional state
- H04L 51/04: Real-time or near real-time messaging, e.g. instant messaging [IM]
- H04M 3/5133: {Operator terminal details}
- H04M 2203/408: Customer-specific call routing plans
- H04M 2203/556: Statistical analysis and interpretation
- H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08)
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

A system includes one or more memory devices storing instructions, and one or more processors configured to execute the instructions to perform steps of a method for providing customer sentiment depiction. The system may receive customer information and session information and generate a customer sentiment estimate. The system may then receive an indication of a detected customer voice characteristic and generate an updated customer sentiment estimate that may be transmitted to a customer service terminal for display.

## Claims

### Claim 1

A system for providing customer sentiment depiction, comprising:
an environmental data sensor for obtaining audio or visual data;
an input/output device;
a display;
one or more processors including at least a sentiment depiction processor; and
a memory in communication with the one or more processors and storing instructions that, when executed by the one or more processors, are configured to cause the system to:
responsive to a customer call being made to an interactive voice response system, receive customer information and session information;
obtain, by the environmental data sensor, customer audio information;
determine, by the sentiment depiction processor, a customer voice characteristic based on the customer audio information;
generate, based on the customer information, the session information, and the customer voice characteristic, a customer sentiment estimate for display by a customer service terminal, wherein the customer sentiment estimate is indicative of a sentiment of a customer engaging with the interactive voice response system;
automatically modify, based on the customer sentiment estimate, (i) a position of the customer call in a queue to speak to a customer service agent and, (ii) a script for the customer service agent to interact with the customer; and
transmit, via the input/output device, the customer sentiment estimate to a call center server for display.

### Claim 2 (depends on 1)

The system of claim 1, wherein the customer sentiment estimate represents an estimated level of happiness according to a predetermined scale.

### Claim 3 (depends on 1)

The system of claim 1, wherein generating the customer sentiment estimate is performed by using predictive modeling.

### Claim 4 (depends on 1)

The system of claim 1, wherein the customer voice characteristic includes one or more of a customer accent, a customer tone of voice, a customer rate of speech, a customer volume level, a customer diction, or a particular word or a phrase spoken by the customer.

### Claim 5

A system for providing customer sentiment depiction, comprising:
one or more processors; and
a memory in communication with the one or more processors and storing instructions that, when executed by the one or more processors, are configured to cause the system to:
responsive to a customer call being made to an interactive voice response system, receive customer information;
generate, based on the customer information, a customer sentiment estimate for display by a customer service terminal, the customer sentiment estimate being indicative of a sentiment of a customer engaging with the interactive voice response system;
receive an indication of a customer voice characteristic;
generate, based on the customer sentiment estimate and the indication of the customer voice characteristic, an updated customer sentiment estimate;
automatically modify, based on the updated customer sentiment estimate and while the customer call is being made, (i) a position of the customer call in a queue to speak to a customer service agent and, (ii) a script for the customer service agent to interact with the customer; and
transmit, to the customer service terminal for display, the updated customer sentiment estimate.

### Claim 6 (depends on 5)

The system of claim 5, wherein the customer sentiment estimate for display includes information relating to past transactions, past interactions, or business accounts associated with the customer.

### Claim 7 (depends on 5)

The system of claim 5, wherein the customer voice characteristic includes one or more of a customer accent, a customer tone of voice, a customer rate of speech, a customer volume level, a customer diction, or a particular word or a phrase spoken by the customer.

### Claim 8 (depends on 5)

The system of claim 5, wherein the customer sentiment estimate represents an estimated level of happiness according to a predetermined scale.

### Claim 9 (depends on 5)

The system of claim 5, wherein generating the customer sentiment estimate is performed using predictive modeling.

### Claim 10 (depends on 5)

The system of claim 5, wherein the instructions further cause the system to:
determine a customer accent from the customer voice characteristic;
output instructions to modify a customer experience in response to the customer accent; and
automatically route, based on the customer accent, the customer call to a particular customer service agent or to a particular group of customer service agents.

### Claim 11

A system for providing customer sentiment depiction, comprising:
one or more processors; and
a memory in communication with the one or more processors and storing instructions that, when executed by the one or more processors, are configured to cause the system to:
receive customer information and session information;
generate, based on the customer information and the session information, a first customer sentiment estimate for display by a customer service terminal, the first customer sentiment estimate being indicative of a sentiment of a customer engaging with a customer service system;
receive an indication of a customer action or a customer voice characteristic;
generate, based on the indication of the customer action or the customer voice characteristic, a second customer sentiment estimate for display by the customer service terminal, the second customer sentiment estimate representing an update to the first customer sentiment estimate;
automatically route, based on an accent of the customer determined from the customer voice characteristic, a customer call to a particular customer service agent or to a particular group of customer service agents;
automatically modify, based on the second customer sentiment estimate and while the customer call is being made, a script for interacting with the customer; and
output for display, to the customer service terminal of the particular customer service agent or the particular group of customer service agents, an indication of the second customer sentiment estimate.

### Claim 12 (depends on 11)

The system of claim 11, wherein the system receives the customer information and the session information in response to a customer interaction with an automated customer service system.

### Claim 13 (depends on 11)

The system of claim 11, wherein the customer information comprises one or more of car loan account information, mortgage account information, and credit card account information.

### Claim 14 (depends on 11)

The system of claim 11, wherein the customer information comprises one or more of a name, birthday, age, sex, nationality, and address associated with the customer information.

### Claim 15 (depends on 11)

The system of claim 11, wherein the customer information comprises one or more of past surveys completed by the customer, previous complaints submitted by the customer, or details of previous interactions between the customer and a company.

### Claim 16 (depends on 11)

The system of claim 11, wherein the session information includes an amount of time that the customer has been engaging with the customer service system for a current session.

### Claim 17 (depends on 11)

The system of claim 11, wherein engaging with the customer service system includes one or more of interacting with an interactive voice response system via a telephone or interacting with an automated chat system via a website or mobile application.

### Claim 18 (depends on 11)

The system of claim 11, wherein the sentiment represents a customer happiness rating associated with the customer according to a predetermined scale.

### Claim 19 (depends on 11)

The system of claim 11, wherein the customer action or the customer voice characteristic comprises one or more a tone of voice, a rate of speech or typing, a volume of voice, diction, or a word or phrase communicated by a customer associated with the customer information.