# US 11,445,068 B1 · Virtual caller system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,445,068 B1
- Document ID: US11445068B1
- Status: Granted
- Kind: B1
- Application number: 16/798,082
- Filed: 2020-02-21
- Granted: 2022-09-13
- Earliest claimed: 2020-02-21
- Assignees: Express Scripts Strategic Development, Inc.
- Applicants: Express Scripts Strategic Development, Inc.
- Inventors: Christopher M. Myers; Danielle L. Smith
- Web page: https://patentgrep.com/patent/US11445068B1

## CPC

- H04M 3/5233: {Operator skill based call distribution}
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/049: Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
- G06N 3/09: Supervised learning
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- H04M 3/493: Interactive information services, e.g. directory enquiries {; Arrangements therefor, e.g. interactive voice response [IVR] systems or voice portals}
- H04M 3/4933: {with operator assistance}
- H04M 3/4936: {Speech interaction details (speech recognition per se G10L15/00)}
- H04M 3/5175: {Call or contact centers supervision arrangements}
- H04M 3/5191: {interacting with the Internet}
- H04M 3/5237: {Interconnection arrangements between ACD systems}
- H04M 11/10: with dictation recording and playback systems
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/08: Learning methods
- H04M 3/523: with call distribution or queueing
- H04M 2201/40: using speech recognition
- H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

Method starts with a processor receiving configuration settings including an identified task, a relationship data, and a criticality value. Processor initializes a communication session with an agent client device. The communication session is between a virtual caller associated with the system and the agent client device. Processor then processes an audio signal of the communication session to generate an agent utterance and generates a transcribed agent utterance based on the agent utterance using a speech-to-text processor. Processor generates a virtual caller utterance using a task-specific virtual caller neural network associated with the identified task. The virtual caller utterance can be generated based on the transcribed agent utterance. Processor then causes the virtual caller utterance to be played back in the communication session to the agent client device. Other embodiments are disclosed herein.

## Claims

### Claim 1

A system comprising:
a processor; and
a memory component having instructions stored thereon, when executed by the processor, causes the processor to perform operations comprising:
receiving a plurality of configuration settings, wherein the configuration settings include an identified task, a relationship data, and a criticality value;
initializing a communication session with an agent client device, wherein the communication session is between a virtual caller associated with the system and the agent client device, wherein the identified task includes a task to be completed by the virtual caller during the communication session, wherein the relationship data includes at least one potential relationship between the virtual caller and a patient that is the subject of the identified task;
processing an audio signal of the communication session to generate an agent utterance;
generating a transcribed agent utterance based on the agent utterance using a speech-to-text processor;
selecting, from a plurality of task-specific virtual caller neural networks, a task-specific virtual caller neural network that is associated with the identified task;
generating a virtual caller utterance using the task-specific virtual caller neural network associated with the identified task, wherein generating the virtual caller utterance is based on the transcribed agent utterance; and
causing the virtual caller utterance to be played back in the communication session to the agent client device.

### Claim 2 (depends on 1)

The system of claim 1, wherein generating the virtual caller utterance is further based on the criticality value.

### Claim 3 (depends on 1)

The system of claim 1, wherein generating the virtual caller utterance is further based on the relationship data.

### Claim 4 (depends on 1)

The system of claim 1, wherein the processor to perform operations further comprising:
generating a status value based on the identified task, wherein generating the virtual caller utterance is further based on the status value.

### Claim 5 (depends on 1)

The system of claim 1, wherein the criticality value that indicates a level of criticality associated with the identified task.

### Claim 6 (depends on 1)

The system of claim 1, wherein the communication session includes a voice call using a telephone system or voice over IP (VoIP).

### Claim 7 (depends on 1)

The system of claim 1, wherein the task-specific virtual caller neural network that is associated with the identified task is a bi-directional long short-term memory (LSTM) that receives the transcribed agent utterance as an input.

### Claim 8 (depends on 1)

The system of claim 1, wherein the processor to perform operations further comprising:
generating an agent action ranking score associated with the virtual caller utterance;
generating a feedback result based on the agent action ranking score; and
causing the feedback result to be displayed on a display device of the agent client device.

### Claim 9

A method comprising:
receiving a plurality of configuration settings, wherein the configuration settings include an identified task, a relationship data, and a criticality value;
initializing a communication session with an agent client device, wherein the communication session is between a virtual caller associated with a virtual caller system and the agent client device, wherein the identified task includes a task to be completed by the virtual caller during the communication session, wherein the relationship data includes at least one potential relationship between the virtual caller and a patient that is the subject of the identified task;
processing an audio signal of the communication session to generate an agent utterance;
generating a transcribed agent utterance based on the agent utterance using a speech-to-text processor;
selecting, from a plurality of task-specific virtual caller neural networks, a task-specific virtual caller neural network that is associated with the identified task;
generating a virtual caller utterance using the task-specific virtual caller neural network associated with the identified task, wherein generating the virtual caller utterance is based on the transcribed agent utterance; and
causing the virtual caller utterance to be played back in the communication session to the agent client device.

### Claim 10 (depends on 9)

The method of claim 9, wherein generating the virtual caller utterance is further based on the criticality value.

### Claim 11 (depends on 9)

The method of claim 9, wherein generating the virtual caller utterance is further based on the relationship data.

### Claim 12 (depends on 9)

The method of claim 9, further comprising:
generating a status value based on the identified task, wherein generating the virtual caller utterance is further based on the status value.

### Claim 13 (depends on 9)

The method of claim 9, wherein the criticality value that indicates a level of criticality associated with the identified task.

### Claim 14 (depends on 9)

The method of claim 9, wherein the communication session includes a voice call using a telephone system or voice over IP (VoIP).

### Claim 15 (depends on 9)

The method of claim 9, wherein the task-specific virtual caller neural network that is associated with the identified task is a bi-directional long short-term memory (LSTM) that receives the transcribed agent utterance as an input.

### Claim 16 (depends on 9)

The method of claim 9, further comprising:
generating an agent action ranking score associated with the virtual caller utterance;
generating a feedback result based on the agent action ranking score; and
causing the feedback result to be displayed on a display device of the agent client device.

### Claim 17

A non-transitory storage medium having instructions stored thereon, when executed by a processor causes the processor to perform operations comprising:
receiving a plurality of configuration settings, wherein the configuration settings include an identified task, a relationship data, and a criticality value;
initializing a communication session with an agent client device, wherein the communication session is between a virtual caller and the agent client device, wherein the identified task includes a task to be completed by the virtual caller during the communication session, wherein the relationship data includes at least one potential relationship between the virtual caller and a patient that is the subject of the identified task;
processing an audio signal of the communication session to generate an agent utterance;
generating a transcribed agent utterance based on the agent utterance using a speech-to-text processor;
selecting, from a plurality of task-specific virtual caller neural networks, a task-specific virtual caller neural network that is associated with the identified task;
generating a virtual caller utterance using the task-specific virtual caller neural network associated with the identified task, wherein generating the virtual caller utterance is based on the transcribed agent utterance; and
causing the virtual caller utterance to be played back in the communication session to the agent client device.

### Claim 18 (depends on 17)

The non-transitory storage medium of claim 17, wherein generating the virtual caller utterance is further based on the criticality value, the relationship data, and a status value.