# US 12,217,752 B2 · System method and apparatus for combining words and behaviors

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,217,752 B2
- Document ID: US12217752B2
- Status: Granted
- Kind: B2
- Application number: 17/749,677
- Filed: 2022-05-20
- Published: 2022-11-24
- Granted: 2025-02-04
- Earliest claimed: 2021-05-21
- Assignees: Verint Americas Inc.
- Applicants: Verint Americas Inc.
- Inventors: John Kane
- Web page: https://patentgrep.com/patent/US12217752B2

## CPC

- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- H04M 3/5175: {Call or contact centers supervision arrangements}
- G06F 40/186: Templates
- G06F 40/279: Recognition of textual entities
- G06F 40/30: Semantic analysis
- G10L 15/18: using natural language modelling
- G10L 15/30: Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
- G10L 25/63: for estimating an emotional state
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- G10L 2015/225: {Feedback of the input speech}
- H04M 3/5183: {Call or contact centers with computer-telephony arrangements}
- H04M 2201/40: using speech recognition
- H04M 2203/401: Performance feedback
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A system and method for integrating audio data collected, such as audio data and analytical data, to perform behavioral analysis on the audio data, using an application of acoustic signal processing and machine learning algorithms, by converting the audio data to text data and performing behavioral analysis on the text data. The behavioral analysis data from the audio application of acoustic signal processing is combined with machine learning algorithms and speech to text data to provide a call agent with feedback to assist in the next best action or insight into customer behaviors.

## Claims

### Claim 1

A computer-implemented method for outputting feedback data to a selected device, the method comprising:
using at least one hardware processor for extracting code for:
accessing audio data from a communication session between a caller and an agent;
generating behavioral data by performing behavioral analysis of at least a portion of the audio data, the behavioral analysis including acoustic signal processing of the portion of audio data and applying one or more machine learning algorithms to the portion of the audio data;
generating text data by converting at least a portion of the audio data to text data;
analyzing the text data using natural language processing;
generating integrated data by combining the behavioral data with the analyzed text data;
accessing rules of a set of rules;
comparing the integrated data to the rules;
generating feedback data based on the comparison; and
outputting the feedback data to the selected device during the communication session between the caller and the agent.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, further comprising:
identifying one or more parameters of the audio data; and
utilizing one or more of the parameters during the generation of the behavioral data and the generation of the text data.

### Claim 3 (depends on 2)

The computer-implemented method of claim 2, wherein the parameters include indicators of an emotional state of a caller.

### Claim 4 (depends on 1)

The computer-implemented method of claim 1, wherein the selected device is a supervisory device.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein the selected device is an agent device.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, wherein the selected device is located at a remote location.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, further comprising:
utilizing the feedback data to affect a subsequent communication session.

### Claim 8

A system for outputting feedback data to a selected device, comprising:
one or more memories configured to store representations of data in an electronic form; and
one or more processors, operatively coupled to one or more of the memories, the processors configured to access the data and process the data to:
access audio data from a communication session between a caller and an agent;
generate behavioral data by performing behavioral analysis of at least a portion of the audio data, the behavioral analysis including acoustic signal processing of the portion of audio data and applying one or more machine learning algorithms to the portion of the audio data;
store the behavioral data;
generate text data by converting at least a portion of the audio data to text data;
analyze the text data by applying natural language processing to the text data;
store the text data;
combine the stored behavioral data with the stored text data;
compare the combined behavioral data and the text data to one or more rules;
generate feedback data based on the comparison; and
output the feedback data to the selected device during the communication session between the caller and the agent.

### Claim 9 (depends on 8)

The system of claim 8, further comprising:
identifying one or more parameters of the audio data; and
utilizing one or more of the parameters during the generation of the behavioral data and the generation of the text data.

### Claim 10 (depends on 9)

The system of claim 9, wherein the parameters include indicators of an emotional state of a caller.

### Claim 11 (depends on 8)

The system of claim 8, wherein the selected device is a supervisory device.

### Claim 12 (depends on 8)

The system of claim 8, wherein the selected device is an agent device.

### Claim 13 (depends on 8)

The system of claim 8, wherein the selected device is located at a remote location.

### Claim 14 (depends on 8)

The system of claim 8, further comprising:
utilizing the feedback data to affect a subsequent communication session.

### Claim 15 (depends on 8)

The system of claim 8, further comprising: storing the feedback data in a database.

### Claim 16

A computer-implemented method for outputting feedback data to a selected device, the method comprising:
using at least one hardware processor for extracting code for:
accessing audio data from a communication session between a caller and an agent;
identifying one or more acoustic portions of the audio data;
identifying one or more text portions of the acoustic data;
accessing one or more models;
combining the one or more acoustic portion of the audio data and the one or more text portions of the audio data using the one or mode models;
generating feedback data based on the combining; and
outputting the feedback data to the selected device during the communication session between the caller and the agent.

### Claim 17 (depends on 1)

The computer-implemented method of claim 1, wherein the feedback data includes an action for the agent to take during the communication session based in part on the generated integrated data.

### Claim 18 (depends on 17)

The computer-implemented method of claim 17, wherein the action for the agent to take includes speaking louder or faster during the communication session.

### Claim 19 (depends on 8)

The system of claim 8, wherein the feedback data includes an action for the agent to take during the communication session based in part on the generated integrated data.

### Claim 20 (depends on 19)

The system of claim 19, wherein the action for the agent to take includes speaking louder or faster during the communication session.