# US 12,183,344 B1 · Systems and methods for determining a next action based on entities and intents

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,183,344 B1
- Document ID: US12183344B1
- Status: Granted
- Kind: B1
- Application number: 17/456,491
- Filed: 2021-11-24
- Granted: 2024-12-31
- Earliest claimed: 2021-11-24
- Assignees: Wells Fargo Bank, N.A.
- Applicants: Wells Fargo Bank, N.A.
- Inventors: Vinothkumar Venkataraman; Rahul Ignatius; Naveen Gururaja Yeri; Paul Davis
- Web page: https://patentgrep.com/patent/US12183344B1

## CPC

- G10L 15/1822: {Parsing for meaning understanding}
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

Systems, apparatuses, methods, and computer program products are disclosed for predicting an entity and intent based on captured speech. An example method includes capturing speech and converting the speech to text. The example method further includes causing generation of one or more entities and one or more intents based on the speech and the text. The example method further includes determining a next action based on each of the one or more entities and each of the one or more intents.

## Claims

### Claim 1

A method for predicting an entity and intent based on captured speech, the method comprising:
receiving, by a communications circuitry and based on prompts from an Interactive Voice Response (IVR) system, a reply comprising speech during a customer call;
separating, by automated speech recognition circuitry, the speech into a plurality of discrete speech segments, wherein the separating is based on a time each of the plurality of discrete speech segments is received;
extracting, by the automated speech recognition circuitry, acoustic features from each of the plurality of discrete speech segments;
converting, by the automated speech recognition circuitry, each of the plurality of discrete speech segments to a corresponding discrete text segment;
causing, by an entity and intent prediction circuitry, generation of a plurality of entities and a plurality of intents for each of the acoustic features and each corresponding discrete text segment, wherein the plurality of entities and the plurality of intents for each of the acoustic features are generated in parallel with the plurality of entities and the each plurality of intents for each corresponding discrete text segment; and
redirecting, by a call processing circuitry, the customer call from the IVR system to an agent device based on the plurality of entities and the plurality of intents.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
causing, by the entity and intent prediction circuitry, generation of a relevancy score for each intent of the plurality of intents.

### Claim 3 (depends on 2)

The method of claim 2, wherein the redirecting is determined based on every generated relevancy score.

### Claim 4 (depends on 2)

The method of claim 2, further comprising:
ranking, by the entity and intent prediction circuitry, each intent of the plurality of intents based on its corresponding relevancy score; and
classifying, by the entity and intent prediction circuitry, each intent of the plurality of intents as a main intent or a sub-intent based on the ranking.

### Claim 5 (depends on 4)

The method of claim 4, wherein the redirecting is further based on the main intent.

### Claim 6 (depends on 5)

The method of claim 5, further comprising:
determining, by the call processing circuitry, one or more additional actions based on each entity of the one or more plurality of entities and each intent of the plurality of intents classified as sub-intents.

### Claim 7 (depends on 6)

The method of claim 6, further comprising:
in response to receiving an indication that the redirecting is complete, performing the one or more additional actions.

### Claim 8 (depends on 1)

The method of claim 1, wherein each entity of the plurality of entities comprise one or more of a business entity, a product or service entity, business segment, call service entity, call service division or segment, service center, or customer service center.

### Claim 9 (depends on 1)

The method of claim 1, wherein each intent of the plurality of intents comprise a purpose for the customer call.

### Claim 10 (depends on 9)

The method of claim 9, wherein the purpose for the customer call is known or unknown by the customer.

### Claim 11 (depends on 1)

The method of claim 1, wherein the entity and intent prediction circuitry includes a custom named entity recognition model, and wherein generation of the plurality of entities uses the custom named entity recognition model.

### Claim 12 (depends on 1)

The method of claim 1, further comprising:
further comprising, prior to the redirecting, determining, by the entity and intent prediction circuitry, a severity based on the speech and the text, and wherein the redirecting is further based on the severity.

### Claim 13 (depends on 12)

The method of claim 12, wherein the severity is either a regular severity or an escalated severity.

### Claim 14 (depends on 13)

The method of claim 13, wherein, if the severity is an escalated severity, the redirecting includes transferring a call associated with the speech and text to an agent.

### Claim 15

An apparatus for predicting an entity and intent based on captured speech, the apparatus comprising:
communications circuitry configured to receive, based on prompts from an Interactive Voice Response (IVR) system, a reply comprising speech during a customer call;
automated speech recognition circuitry configured to:
separate the speech into a plurality of discrete speech segments, wherein the separating is based on a time each of the plurality of discrete speech segments is received,
extract acoustic features from each of the plurality of discrete speech segments, and
convert each of the plurality of discrete speech segments to a corresponding discrete text segment;
entity and intent prediction circuitry configured to cause generation of a plurality of entities and a plurality of intents for each of the acoustic features and each corresponding discrete text segment, wherein the plurality of entities and the plurality of intents for each of the acoustic features are generated in parallel with the plurality of entities and the plurality of intents for each corresponding discrete text segment; and
call processing circuitry configured to redirect the customer call from the IVR system to an agent device based on each of the plurality of entities and each of the plurality of intents.

### Claim 16 (depends on 15)

The apparatus of claim 15, wherein the entity and intent prediction circuitry causes generation of a relevancy score for each intent of the plurality of intents.

### Claim 17 (depends on 16)

The apparatus of claim 16, wherein the entity and intent prediction circuitry ranks each intent of the plurality of intents based on the relevancy score.

### Claim 18 (depends on 17)

The apparatus of claim 17, wherein the entity and intent prediction circuitry uses a one vs. rest support vector machine model to generate the relevancy score for each intent of the plurality of intents and to rank each intent of the one or more plurality of intents.

### Claim 19

A computer program product for predicting an entity and intent based on captured speech, the computer program product comprising at least one non-transitory computer-readable storage medium storing software instructions that, when executed, cause an apparatus to:
receive, based on prompts from an Interactive Voice Response (IVR) system, a reply comprising speech during a customer call;
separate the speech into a plurality of discrete speech segments, wherein the separating is based on a time each of the plurality of discrete speech segments is received;
extract acoustic features from each of the plurality of discrete speech segments;
convert each of the plurality of discrete speech segments to a corresponding discrete text segment;
cause generation of a plurality of entities and a plurality of intents for each of the acoustic features and each corresponding discrete text segment, wherein the plurality of entities and the plurality of intents for each of the acoustic features are generated in parallel with the plurality of entities and the plurality of intents for each corresponding discrete text segment; and
redirect the customer call from the IVR system to an agent device based on each of the plurality of entities and each of the plurality of intents.

### Claim 20 (depends on 19)

The computer program product of claim 19, wherein the software instructions, when executed, further cause the apparatus to:
cause generation of a severity based on the plurality of discrete speech segments and each corresponding discrete text segment.

### Claim 21 (depends on 19)

The computer program product of claim 19, wherein the redirecting comprises one or more of re-directing the customer call from an IVR menu to another IVR menu, selecting an agent based on the plurality of entities and the plurality of intents, generating a customer's personalized product or service recommendation, re-directing the customer call based on the customer's personalized product or service recommendation, determining an agent's rating or ranking, or generating a call back for a customer.