# US 10,122,855 B1 · Voice and speech recognition for call center feedback and quality assurance

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,122,855 B1
- Document ID: US10122855B1
- Status: Granted
- Kind: B1
- Application number: 15/823,850
- Filed: 2017-11-28
- Granted: 2018-11-06
- Earliest claimed: 2015-06-29
- Assignees: State Farm Mutual Automobile Insurance Company
- Applicants: State Farm Mutual Automobile Insurance Company
- Inventors: Sylvia Hernandez
- Web page: https://patentgrep.com/patent/US10122855B1

## CPC

- H04M 3/5175: {Call or contact centers supervision arrangements}
- G06F 40/205: Parsing
- G10L 17/04: Training, enrolment or model building
- G10L 17/08: Use of distortion metrics or a particular distance between probe pattern and reference templates
- G10L 25/51: for comparison or discrimination
- G10L 25/63: for estimating an emotional state
- H04M 3/42: Systems providing special services or facilities to subscribers (specially adapted for wireless communication networks H04W4/00)
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- G10L 25/87: Detection of discrete points within a voice signal
- H04M 2201/40: using speech recognition
- H04M 2201/41: using speaker recognition
- H04M 2203/40: related to call centers
- H04M 2203/401: Performance feedback
- H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

A computer-implemented method for providing an objective evaluation to a customer service representative regarding his performance during an interaction with a customer may include receiving a digitized data stream corresponding to a spoken conversation between a customer and a representative; converting the data stream to a text stream; generating a representative transcript that includes the words from the text stream that are spoken by the representative; comparing the representative transcript with a plurality of positive words and a plurality of negative words; and generating a score that varies according to the occurrence of each word spoken by the representative that matches one of the positive words, and/or the occurrence of each word spoken by the representative that matches one of the negative words. Tone of voice, as well as response time, during the interaction may also be monitored and analyzed to adjust the score, or generate a separate score.

## Claims

### Claim 1

A computer-implemented method for providing an objective evaluation to a customer service representative regarding his performance during an interaction with a customer, the computer-implemented method comprising:
comparing, via one or more processors, a customer service representative transcript that includes words which are spoken by the customer service representative during an oral conversation between the customer service representative and the customer with a plurality of positive words and a plurality of negative words; and
calculating and displaying on a display, via the one or more processors, a first score that varies according to the occurrence of each word spoken by the customer service representative that matches one of the positive words and the occurrence of each word spoken by the customer service representative that matches one of the negative words to facilitate an objective evaluation of the customer interaction such that the first score is increased corresponding to usage of positive words and decreased corresponding to usage of negative words.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, further comprising:
receiving, via one or more processors, a digitized data stream corresponding to the oral conversation between the customer and the customer service representative; and
converting, via the one or more processors, the data stream to a text stream.

### Claim 3 (depends on 2)

The computer-implemented method of claim 2, further comprising:
generating, via the one or more processors, a plurality of voice prints, each voice print derived from one of a plurality of periods of time during the conversation,
comparing, via the one or more processors, each voice print with a voice print of the customer service representative to determine an identity of the voice print,
indicating, via the one or more processors, a first set of voice prints that are associated with the customer service representative and a second set of voice prints that are associated with the customer,
matching, via the one or more processors, the first set of voice prints with words from the text stream that are spoken by the customer service representative, and
displaying, via the one or more processors, on the display a list of words from the customer service representative transcript that match one or more positive words, and a list of words from the customer service representative transcript that match one or more negative words.

### Claim 4 (depends on 2)

The computer-implemented method of claim 2, further comprising:
analyzing, via the one or more processors, the data stream to determine a plurality of tone of voice values for the customer service representative, each tone of voice value derived from one of a plurality of periods of time during the conversation;
analyzing, via the one or more processors, the data stream to determine a plurality of tone of voice values for the customer, each tone of voice value derived from one of a plurality of periods of time during the conversation; and
generating and displaying on the display, via the one or more processors, a second score that varies according to whether the customer service representative changed his tone of voice in response to a change in tone of voice of the customer to facilitate the objective evaluation of a customer interaction.

### Claim 5 (depends on 4)

The computer-implemented method of claim 4, further comprising determining, via the one or more processors, each occurrence when the customer's tone of voice value is above a first threshold and determining, via the one or more processors, whether the tone of voice value for the customer service representative increases above a second threshold within a first time period after each occurrence.

### Claim 6 (depends on 5)

The computer-implemented method of claim 5, further comprising
increasing, via the one or more processors, the second score for each occurrence when the tone of voice value for the customer service representative does not increase above the second threshold within the first time period;
decreasing, via the one or more processors, the second score for each occurrence when the tone of voice value for the customer service representative does increase above the second threshold within the first time period; and
displaying, via the one or more processors, on the display the second score and a list of occurrences when the customer service representative changed his tone of voice in response to a change in tone of voice of the customer.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, further comprising:
identifying, via the one or more processors, a voice of the customer service representative and a voice of the customer;
determining, via the one or more processors, when the customer stops talking;
determining, via the one or more processors, when the customer service representative starts talking thereafter;
determining, via the one or more processors, a response time corresponding to a period of time that elapses between when the customer stops talking and the customer service representative starts talking;
generating and displaying, via the one or more processors, a third score that varies according to a value of the response time;
decreasing, via the one or more processors, the third score for every occurrence when the response time is less than a lower threshold or greater than an upper threshold; and
displaying, via the one or more processors, on the display a list of occurrences when the response time is less than a lower threshold or greater than an upper threshold.

### Claim 8

A computing device for providing an objective evaluation to a customer service representative regarding his performance during an interaction with a customer, the computing device comprising:
a processing element in electronic communication with a memory element, the processing element configured to
compare a customer service representative transcript that includes words which are spoken by the customer service representative during an oral conversation between the customer service representative and the customer with a plurality of positive words and a plurality of negative words, and
calculate and display on a display a first score that varies according to the occurrence of each word spoken by the customer service representative that matches one of the positive words and the occurrence of each word spoken by the customer service representative that matches one of the negative words to facilitate an objective evaluation of the customer interaction such that the first score is increased corresponding to usage of positive words and decreased corresponding to usage of negative words.

### Claim 9 (depends on 8)

The computing device of claim 8, wherein the processing element is further configured to receive a digitized data stream corresponding to the oral conversation between the customer and the customer service representative, and convert the data stream to a text stream.

### Claim 10 (depends on 9)

The computing device of claim 9, wherein the processing element is further configured to
generate a plurality of voice prints, each voice print derived from one of a plurality of periods of time during the conversation,
compare each voice print with a voice print of the customer service representative to determine an identity of the voice print,
indicate a first set of voice prints that are associated with the customer service representative and a second set of voice prints that are associated with the customer,
match the first set of voice prints with words from the text stream that are spoken by the customer service representative, and
display on the display a list of words from the customer service representative transcript that match one or more positive words, and a list of words from the customer service representative transcript that match one or more negative words.

### Claim 11 (depends on 9)

The computing device of claim 9, wherein the processing element is further configured to
analyze the data stream to determine a plurality of tone of voice values for the customer service representative, each tone of voice value derived from one of a plurality of periods of time during the conversation;
analyze the data stream to determine a plurality of tone of voice values for the customer, each tone of voice value derived from one of a plurality of periods of time during the conversation; and
generate and display on the display a second score that varies according to whether the customer service representative changed his tone of voice in response to a change in tone of voice of the customer to facilitate the objective evaluation of a customer interaction.

### Claim 12 (depends on 11)

The computing device of claim 11, wherein the processing element is further configured to determine each occurrence when the customer's tone of voice value is above a first threshold and determine whether the tone of voice value for the customer service representative increases above a second threshold within a first time period after each occurrence.

### Claim 13 (depends on 12)

The computing device of claim 12, wherein the processing element is further configured to
increase the second score for each occurrence when the tone of voice value for the customer service representative does not increase above the second threshold within the first time period,
decrease the second score for each occurrence when the tone of voice value for the customer service representative does increase above the second threshold within the first time period, and
display on the display the second score and a list of occurrences when the customer service representative changed his tone of voice in response to a change in tone of voice of the customer.

### Claim 14 (depends on 8)

The computing device of claim 8, wherein the processing element is further configured to
identify a voice of the customer service representative and a voice of the customer,
determine when the customer stops talking,
determine when the customer service representative starts talking thereafter,
determine a response time corresponding to a period of time that elapses between when the customer stops talking and the customer service representative starts talking,
generate and display a third score that varies according to a value of the response time,
decrease the third score for every occurrence when the response time is less than a lower threshold or greater than an upper threshold, and
display on the display a list of occurrences when the response time is less than a lower threshold or greater than an upper threshold.

### Claim 15

A non-transitory computer-readable medium with an executable program stored thereon for providing an evaluation to a customer service representative regarding his performance during an interaction with a customer, wherein the program instructs a processing element of a computing device to perform the following:
comparing a customer service representative transcript that includes words which are spoken by the customer service representative during an oral conversation between the customer service representative and the customer with a plurality of positive words and a plurality of negative words; and
calculating and displaying on a display a first score that varies according to the occurrence of each word spoken by the customer service representative that matches one of the positive words and the occurrence of each word spoken by the customer service representative that matches one of the negative words to facilitate an objective evaluation of the customer interaction such that the first score is increased corresponding to usage of positive words and decreased corresponding to usage of negative words.

### Claim 16 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein the program further instructs the processing element to receive a digitized data stream corresponding to the oral conversation between the customer and the customer service representative, and convert the data stream to a text stream.

### Claim 17 (depends on 16)

The non-transitory computer-readable medium of claim 16, wherein the program further instructs the processing element to
generate a plurality of voice prints, each voice print derived from one of a plurality of periods of time during the conversation,
compare each voice print with a voice print of the customer service representative to determine an identity of the voice print,
indicate a first set of voice prints that are associated with the customer service representative and a second set of voice prints that are associated with the customer,
match the first set of voice prints with words from the text stream that are spoken by the customer service representative, and
display on the display a list of words from the customer service representative transcript that match one or more positive words, and a list of words from the customer service representative transcript that match one or more negative words.

### Claim 18 (depends on 16)

The non-transitory computer-readable medium of claim 16, wherein the program further instructs the processing element to
analyze the data stream to determine a plurality of tone of voice values for the customer service representative, each tone of voice value derived from one of a plurality of periods of time during the conversation,
analyze the data stream to determine a plurality of tone of voice values for the customer, each tone of voice value derived from one of a plurality of periods of time during the conversation,
generate and display on the display a second score that varies according to whether the customer service representative changed his tone of voice in response to a change in tone of voice of the customer to facilitate the objective evaluation of a customer interaction, and
determine each occurrence when the customer's tone of voice value is above a first threshold and determine whether the tone of voice value for the customer service representative increases above a second threshold within a first time period after each occurrence.

### Claim 19 (depends on 18)

The non-transitory computer-readable medium of claim 18, wherein the program further instructs the processing element to
increase the second score for each occurrence when the tone of voice value for the customer service representative does not increase above the second threshold within the first time period,
decrease the second score for each occurrence when the tone of voice value for the customer service representative does increase above the second threshold within the first time period, and
display on the display the second score and a list of occurrences when the customer service representative changed his tone of voice in response to a change in tone of voice of the customer.

### Claim 20 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein the program further instructs the processing element to
identify a voice of the customer service representative and a voice of the customer,
determine when the customer stops talking,
determine when the customer service representative starts talking thereafter,
determine a response time corresponding to a period of time that elapses between when the customer stops talking and the customer service representative starts talking,
generate and display a third score that varies according to a value of the response time,
decrease the third score for every occurrence when the response time is less than a lower threshold or greater than an upper threshold, and
display on the display a list of occurrences when the response time is less than a lower threshold or greater than an upper threshold.