# US 12,118,523 B2 · Virtual assistant host platform configured for interactive voice response simulation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,118,523 B2
- Document ID: US12118523B2
- Status: Granted
- Kind: B2
- Application number: 18/215,875
- Filed: 2023-06-29
- Published: 2023-10-19
- Granted: 2024-10-15
- Earliest claimed: 2020-07-20
- Assignees: Bank of America Corporation
- Applicants: Bank of America Corporation
- Inventors: Srinivas Dundigalla; Saurabh Mehta; Pavan K. Chayanam
- Web page: https://patentgrep.com/patent/US12118523B2

## CPC

- G06Q 20/108: {Remote banking, e.g. home banking}
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G06Q 20/3223: {Realising banking transactions through M-devices}
- G06Q 20/3267: {In-app payments}
- G06Q 20/401: {Transaction verification}
- G06Q 20/4012: {Verifying personal identification numbers [PIN]}
- G06Q 20/40145: {Biometric identity checks}
- G06Q 40/03: Credit; Loans; Processing thereof
- G10L 13/00: Speech synthesis; Text to speech systems
- G10L 15/063: {Training}
- G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning}
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L 2015/223: {Execution procedure of a spoken command}
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

Aspects of the disclosure relate to using machine learning to simulate an interactive voice response system. A computing platform may establish a virtual assistant session with a mobile banking application executing on a mobile device, which may include authenticating at least one authentication credential associated with an online banking account. The computing platform may receive an assistance message from the mobile device requesting assistance. Using a machine learning model, the computing platform may identify an intent of the assistance message. The computing platform may generate a response message based on the intent of the assistance message. The computing platform may send the response message and one or more commands directing the mobile device to output an audio response file based on the response message to the mobile device, which may cause the mobile device to convert the response message into the audio response file and output the audio response file.

## Claims

### Claim 1

A computing platform, comprising:
at least one processor;
a communication interface communicatively coupled to the at least one processor; and
memory storing computer-readable instructions that, when executed by the at least one processor, cause the computing platform to:
train a machine learning model to identify user intent during a virtual interactive voice response session, wherein training the machine learning model comprises:
inputting previously recorded audio files into the machine learning model, wherein the previously recorded audio files are labelled with corresponding identified intents;
receive, from an enterprise computing device, user interaction data, indicating interactions between a first user and the enterprise computing device;
receive, from a second enterprise computing device, second user interaction data, indicating interactions between a second user and the second enterprise computing device;
based on an identifier of the first user, identifying the user interaction data;
generate, based on the identified user interaction data, hotkey information corresponding to the first user, wherein the hotkey information indicates a predicted request for assistance, wherein generating the hotkey information for the first user is further based in part on the second user interaction data;
send, to a mobile device, different than the enterprise computing device, one or more commands directing the mobile device to output a hotkey corresponding to the hotkey information, wherein sending the one or more commands directing the mobile device to output the hotkey causes the mobile device to output the hotkey;
receive an assistance message from the mobile device requesting assistance, wherein the assistance message was sent based on selection of the hotkey;
identify, using the machine learning model, an intent of the assistance message;
generate a response message based on the intent of the assistance message;
send the response message and one or more commands directing the mobile device to output an audio response file based on the response message to the mobile device, wherein sending the response message and the one or more commands directing the mobile device to output the audio response file based on the response message causes the mobile device to convert the response message into the audio response file and output the audio response file;
receive, from the mobile device, feedback information indicating whether or not the audio response file provided the requested assistance; and
update, using the feedback information, the machine learning model.

### Claim 2 (depends on 1)

The computing platform of claim 1, wherein the memory stores additional computer-readable instructions that, when executed by the at least one processor, further cause the computing platform to:
establish a virtual assistant session with a mobile banking application executing on a mobile device, wherein establishing the virtual assistant session with the mobile banking application executing on the mobile device comprises authenticating at least one authentication credential associated with an online banking account.

### Claim 3 (depends on 2)

The computing platform of claim 2, wherein authenticating the at least one authentication credential associated with the online banking account comprises authenticating one or more of: a user name, a password, biometric information, or a voice input.

### Claim 4 (depends on 2)

The computing platform of claim 2, wherein the virtual assistant session corresponds to a data channel between the mobile device and an artificial intelligence engine hosted by the computing platform.

### Claim 5 (depends on 2)

The computing platform of claim 2, wherein the memory stores additional computer-readable instructions that, when executed by the at least one processor, cause the computing platform to:
receive, after establishing the virtual assistant session, additional user interaction data, indicating additional interactions between the first user and the enterprise computing device;
establish a new virtual assistant session with the mobile banking application executing on a mobile device;
based on an identifier of the first user, identify the additional user interaction data, wherein the additional user interaction data indicates a new event unrelated to the user interaction data; and
generate, based on the additional user interaction data, new hotkey information corresponding to the first user, wherein the new hotkey information indicates a new predicted request for assistance, different than the predicted request for assistance, corresponding to the new event.

### Claim 6 (depends on 1)

The computing platform of claim 1, wherein the memory stores additional computer-readable instructions that, when executed by the at least one processor, further cause the computing platform to:
prior to identifying the intent, train the machine learning model based on a plurality of recorded interactive voice response (IVR) sessions, wherein the plurality of recorded interactive voice response sessions correspond to one or more of: phone sessions or virtual IVR sessions with one or more online banking customers.

### Claim 7 (depends on 1)

The computing platform of claim 1, wherein identifying the intent comprises identifying one or more of: a balance inquiry request, a transaction status request, a request for information corresponding to a failed transaction, a credit score inquiry, credit card information, charge/payment information, a request for account information, a mortgage request, a request to execute a transaction, or a request for outage information.

### Claim 8 (depends on 1)

The computing platform of claim 1, wherein generating the response message based on the intent of the assistance message comprises:
sending one or more commands directing an event processing system to process an event based on the intent of the assistance message, wherein sending the one or more commands directing the event processing system to process the event based on the intent of the assistance message cause the event processing system to process the event;
receiving, from the event processing system, an event processing notification indicating that the event was processed; and
generating, based on the event processing notification indicating that the event was processed, the assistance message.

### Claim 9 (depends on 1)

The computing platform of claim 1, wherein identifying the intent of the assistance message comprises identifying the intent of the assistance message based at least in part on information from the mobile device.

### Claim 10

A method comprising:
at a computing platform comprising at least one processor, a communication interface, and memory:
training a machine learning model to identify user intent during a virtual interactive voice response session, wherein training the machine learning model comprises:
inputting previously recorded audio files into the machine learning model, wherein the previously recorded audio files are labelled with corresponding identified intents;
receiving, from an enterprise computing device, user interaction data, indicating interactions between a first user and the enterprise computing device;
receiving, from a second enterprise computing device, second user interaction data, indicating interactions between a second user and the second enterprise computing device;
based on an identifier of the first user, identifying the user interaction data;
generate, based on the identified user interaction data, hotkey information corresponding to the first user, wherein the hotkey information indicates a predicted request for assistance, wherein generating the hotkey information for the first user is further based in part on the second user interaction data;
send, to a mobile device, different than the enterprise computing device, one or more commands directing the mobile device to output a hotkey corresponding to the hotkey information, wherein sending the one or more commands directing the mobile device to output the hotkey causes the mobile device to output the hotkey;
receiving an assistance message from the mobile device requesting assistance, wherein the assistance message was sent based on selection of the hotkey;
identifying, using the machine learning model, an intent of the assistance message;
generating a response message based on the intent of the assistance message;
sending the response message and one or more commands directing the mobile device to output an audio response file based on the response message to the mobile device, wherein sending the response message and the one or more commands directing the mobile device to output the audio response file based on the response message causes the mobile device to convert the response message into the audio response file and output the audio response file;
receiving, from the mobile device, feedback information indicating whether or not the audio response file provided the requested assistance; and
updating, using the feedback information, the machine learning model.

### Claim 11 (depends on 10)

The method of claim 10, further comprising:
establishing a virtual assistant session with a mobile banking application executing on a mobile device, wherein establishing the virtual assistant session with the mobile banking application executing on the mobile device comprises authenticating at least one authentication credential associated with an online banking account.

### Claim 12 (depends on 11)

The method of claim 11, wherein authenticating the at least one authentication credential associated with the online banking account comprises authenticating one or more of: a user name, a password, biometric information, or a voice input.

### Claim 13 (depends on 11)

The method of claim 11, further comprising:
receiving, after establishing the virtual assistant session, additional user interaction data, indicating additional interactions between the first user and the enterprise computing device;
establishing a new virtual assistant session with the mobile banking application executing on a mobile device;
based on an identifier of the first user, identifying the additional user interaction data, wherein the additional user interaction data indicates a new event unrelated to the user interaction data; and
generating, based on the additional user interaction data, new hotkey information corresponding to the first user, wherein the new hotkey information indicates a new predicted request for assistance, different than the predicted request for assistance, corresponding to the new event.

### Claim 14 (depends on 10)

The method of claim 10, further comprising:
prior to identifying the intent, training the machine learning model based on a plurality of recorded interactive voice response (IVR) sessions, wherein the plurality of recorded interactive voice response sessions correspond to one or more of: phone sessions or virtual IVR sessions with one or more online banking customers.

### Claim 15 (depends on 10)

The method of claim 10, wherein identifying the intent comprises identifying one or more of: a balance inquiry request, a transaction status request, a request for information corresponding to a failed transaction, a credit score inquiry, credit card information, charge/payment information, a request for account information, a mortgage request, a request to execute a transaction, or a request for outage information.

### Claim 16 (depends on 10)

The method of claim 10, wherein generating the response message based on the intent of the assistance message comprises:
sending one or more commands directing an event processing system to process an event based on the intent of the assistance message, wherein sending the one or more commands directing the event processing system to process the event based on the intent of the assistance message cause the event processing system to process the event;
receiving, from the event processing system, an event processing notification indicating that the event was processed; and
generating, based on the event processing notification indicating that the event was processed, the assistance message.

### Claim 17 (depends on 10)

The method of claim 10, wherein identifying the intent of the assistance message comprises identifying the intent of the assistance message based at least in part on information from the mobile device.

### Claim 18

One or more non-transitory computer-readable media storing instructions that, when executed by a computing platform comprising at least one processor, a communication interface, and memory, cause the computing platform to:
train a machine learning model to identify user intent during a virtual interactive voice response session, wherein training the machine learning model comprises:
inputting previously recorded audio files into the machine learning model, wherein the previously recorded audio files are labelled with corresponding identified intents;
receive, from an enterprise computing device, user interaction data, indicating interactions between a first user and the enterprise computing device;
receive, from a second enterprise computing device, second user interaction data, indicating interactions between a second user and the second enterprise computing device;
based on an identifier of the first user, identifying the user interaction data;
generate, based on the identified user interaction data, hotkey information corresponding to the first user, wherein the hotkey information indicates a predicted request for assistance, wherein generating the hotkey information for the first user is further based in part on the second user interaction data;
send, to a mobile device, different than the enterprise computing device, one or more commands directing the mobile device to output a hotkey corresponding to the hotkey information, wherein sending the one or more commands directing the mobile device to output the hotkey causes the mobile device to output the hotkey;
receive an assistance message from the mobile device requesting assistance, wherein the assistance message was sent based on selection of the hotkey;
identify, using the machine learning model, an intent of the assistance message;
generate a response message based on the intent of the assistance message; and
send the response message and one or more commands directing the mobile device to output an audio response file based on the response message to the mobile device, wherein sending the response message and the one or more commands directing the mobile device to output the audio response file based on the response message causes the mobile device to convert the response message into the audio response file and output the audio response file;
receive, from the mobile device, feedback information indicating whether or not the audio response file provided the requested assistance; and
update, using the feedback information, the machine learning model.

### Claim 19 (depends on 18)

The one or more non-transitory computer-readable media of claim 18, wherein the memory stores additional computer-readable instructions that, when executed by the at least one processor, further cause the computing platform to:
establish a virtual assistant session with a mobile banking application executing on a mobile device, wherein establishing the virtual assistant session with the mobile banking application executing on the mobile device comprises authenticating at least one authentication credential associated with an online banking account.

### Claim 20 (depends on 19)

The one or more non-transitory computer-readable media of claim 19, wherein authenticating the at least one authentication credential associated with the online banking account comprises authenticating one or more of: a user name, a password, biometric information, or a voice input.