# US 2019/0378519 A1 · System, device and method for enforcing privacy during a communication session with a voice assistant

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0378519 A1
- Document ID: US20190378519A1
- Status: Published application
- Kind: A1
- Application number: 16/144,752
- Filed: 2018-09-27
- Published: 2019-12-12
- Earliest claimed: 2018-06-08
- Assignees: The Toronto-Dominion Bank
- Applicants: The Toronto-Dominion Bank
- Inventors: Milos DUNJIC; David Samuel TAX; Arthur Carroll CHOW; Peter Glen NAIRN; Edward James HOOD; Martin Albert LOZON; John Jong-Suk LEE; Arun Victor JAGGA
- Web page: https://patentgrep.com/patent/US20190378519A1

## CPC

- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G10L 17/22: Interactive procedures; Man-machine interfaces
- G06F 21/32: using biometric data, e.g. fingerprints, iris scans or voiceprints
- G06F 21/606: {by securing the transmission between two devices or processes}
- G06F 21/62: Protecting access to data via a platform, e.g. using keys or access control rules
- G06F 21/83: input devices, e.g. keyboards, mice or controllers thereof
- G10L 17/00: Speaker identification or verification techniques
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

A system, device and method for enforcing privacy during a communication session with a voice assistant are disclosed. In response to a determination that an environment of a first voice assistant device is not private, a first secure communication session between the first voice assistant device and an application server is suspended. In response a determination that one or more other voice assistant devices have been authorized for communication with the application server is made and input to transfer the first secure communication session, a second secure communication session between a second voice assistant device and the application server is initiated. The first secure communication session between the first voice assistant device and the application server is terminated in response to successful initiation of the second secure communication session.

## Claims

### Claim 1

An application server, comprising:
a processor;
a memory coupled to the processor, the memory having stored thereon executable instructions that, when executed by the processor, cause the application server to:
initiate a first secure communication session between a first voice assistant device and the application server for a resource owner, wherein a secure communication session authorizes the application server to access private data in a secure resource and communicate private data from the secure resource to other devices, wherein session data for the first secure communication session is stored by the application server in a secure container;
in response to a determination that an environment of the first voice assistant device is not private:
suspend the first secure communication session between the first voice assistant device and the application server;
determine from an authorization table stored by the application server whether any other voice assistant devices have been authorized for communication with the application server;
in response to a determination that one or more other voice assistant devices have been authorized for communication with the application server:
cause the first voice assistant device to generate a prompt for input whether to transfer the first secure communication session to one of the one or more other voice assistant devices that have been authorized for communication with the application server;
in response to input to transfer the first secure communication session to one of the one or more other voice assistant devices that have been authorized for communication with the application server:
initiate a second secure communication session between a second voice assistant device and the application server; and
terminate the first secure communication session between the first voice assistant device and the application server in response to successful initiation of the second secure communication session between the second voice assistant device and the application server.

### Claim 2 (depends on 1)

The application server of claim 1, wherein the executable instructions, when executed by the processor, further cause the application server to:
before initiating the second secure communication session between the second voice assistant device and the application server:
determine whether a session token for the second voice assistant device is valid;
in response to a determination that the session token for the second voice assistant device is not valid;
cause the second voice assistant device to generate a prompt for input of a shared secret;
determine whether input received in response to the prompt matches the shared secret; and
in response to input matching the shared secret, renew the session token and initiate the second secure communication session between the second voice assistant device and the application server.

### Claim 3 (depends on 1)

The application server of claim 1, wherein the executable instructions to determine from the authorization table stored by the application server whether any other voice assistant devices have been authorized for communication with the application server, when executed by the processor, further cause the application server to:
determine for each of the one or more other voice assistant devices in the authorization table whether an access token is stored by the application server.

### Claim 4 (depends on 3)

The application server of claim 3, wherein the executable instructions to determine for each of the one or more other voice assistant devices in the authorization table whether an access token is stored by the application server, when executed by the processor, further cause the application server to:
determine for each access token stored by the application server whether the access token is valid.

### Claim 5 (depends on 1)

The application server of claim 1, wherein the first secure communication session and second secure communication session are implemented using the OAuth (Open Authorization) 2.0 standard or a successor thereto.

### Claim 6 (depends on 1)

The application server of claim 1, wherein the executable instructions, when executed by the processor, further cause the application server to:
in response to a determination that the one or more other voice assistant devices have been authorized for communication with the application server;
determine from the authorization table stored by the application server a device name for each of the one or more other voice assistant devices;
wherein the prompt generated by the first voice assistant device identifies each of the one or more other voice assistant devices by a respective device name, the prompt being configured to prompt for selection of one of the one or more other voice assistant devices by the respective device name.

### Claim 7 (depends on 6)

The application server of claim 6, wherein the executable instructions, when executed by the processor, further cause the application server to:
in response to input of a device name of an authorized voice assistant device:
initiate the second secure communication session between the voice assistant device identified by the input device name of an authorized voice assistant device and the application server; and
terminate the first secure communication session between the first voice assistant device and the application server in response to successful initiation of the second secure communication session between the voice assistant device identified by the input device name and the application server.

### Claim 8 (depends on 6)

The application server of claim 6, wherein the executable instructions, when executed by the processor, further cause the application server to:
determine for each of the one or more other voice assistant devices that have been authorized for communication with the application server whether an environment in which the respective voice assistant device is located is private;
wherein only devices names for voice assistant devices located in the environment determined to be private are included in the prompt.

### Claim 9 (depends on 1)

The application server of claim 1, wherein the executable instructions, when executed by the processor, further cause the application server to:
determine for each of the one or more other voice assistant devices that have been authorized for communication with the application server whether an environment in which the respective voice assistant device is located is private;
wherein the second secure communication session between the second voice assistant device and the application server is initiated only when the environment in which the second voice assistant device is located is determined to be private.

### Claim 10 (depends on 9)

The application server of claim 9, wherein the executable instructions, when executed by the processor, further cause the application server to:
in response to a determination that none of the one or more other voice assistant devices that have been authorized for communication with the application server is located in an environment that has been determined to be private:
cause the first voice assistant device to generate a prompt for input whether to initiate a call back to a designated telephone number; and
initiate the call back to the designated telephone number in response to input to initiate the call back.

### Claim 11 (depends on 9)

The application server of claim 9, wherein the executable instructions to determine for each of the one or more other voice assistant devices that have been authorized for communication with the application server whether an environment in which the respective voice assistant device is located is private, when executed by the processor, further cause the application server to:
determine a privacy rating for each of the one or more other voice assistant devices that have been authorized for communication with the application server.

### Claim 12 (depends on 11)

The application server of claim 11, wherein the privacy rating is provided by each respective voice assistant device.

### Claim 13 (depends on 11)

The application server of claim 11, wherein the privacy rating is determined by the application server based on sensor data provided by each respective voice assistant device.

### Claim 14 (depends on 9)

The application server of claim 9, wherein the executable instructions to determine for each of the one or more other voice assistant devices that have been authorized for communication with the application server whether an environment in which the respective voice assistant device is located is private, when executed by the processor, further cause the application server to:
receive sensor data by each of the one or more other voice assistant devices that have been authorized for communication with the application server;
process the sensor data to determine, for each of the one or more other voice assistant devices that have been authorized for communication with the application server, whether a person is present in the environment in which the respective voice assistant device is located;
wherein the environment in which a respective voice assistant device is located is determined to be private only when no person is present in the environment in which the respective voice assistant device is located.

### Claim 15 (depends on 9)

The application server of claim 9, wherein the executable instructions to determine for each of the one or more other voice assistant devices that have been authorized for communication with the application server whether an environment in which the respective voice assistant device is located is private, when executed by the processor, further cause the application server to:
receive sensor data by each of the one or more other voice assistant devices that have been authorized for communication with the application server;
process the sensor data to determine, for each of the one or more other voice assistant devices that have been authorized for communication with the application server, whether a person is present in the environment in which the respective voice assistant device is located;
in a response to a determination that at least one person is present in the environment in which a respective voice assistant device is located, determine whether the environment of the respective voice assistant device matches one or more predetermined privacy criteria for a multi-person environment;
wherein the environment in which a respective voice assistant device is located is determined to be private only when the environment of the respective voice assistant device matches the one or more predetermined privacy criteria for the multi-person environment.

### Claim 16 (depends on 15)

The application server of claim 15, wherein the one or more predetermined privacy criteria for the multi-person environment comprises each person other than the resource owner being more than a threshold distance from the second voice assistant device.

### Claim 17 (depends on 15)

The application server of claim 15, wherein the one or more predetermined privacy criteria for the multi-person environment comprises each person other than the resource owner being an authorized user.

### Claim 18 (depends on 1)

The application server of claim 1, wherein the executable instructions, when executed by the processor, further cause the application server to:
periodically during the first communication session determine whether the environment of the first voice assistant device is private.

### Claim 19 (depends on 1)

The application server of claim 1, wherein the executable instructions, when executed by the processor, further cause the application server to:
in response to a determination that no other voice assistant devices have been authorized for communication with the application server:
cause the first voice assistant device to generate a prompt for input whether to initiate a call back to a designated telephone number; and
initiate the call back to the designated telephone number in response to input to initiate the call back.

### Claim 20

A method of transferring a secure communication session between a voice assistant device and an application server, comprising:
initiating a first secure communication session between a first voice assistant device and the application server for a resource owner, wherein a secure communication session authorizes the application server to access private data in a secure resource and communicate private data from the secure resource to other devices, wherein session data for the first secure communication session is stored by the application server in a secure container;
in response to a determination that an environment of the first voice assistant device is not private:
suspending the first secure communication session between the first voice assistant device and the application server;
determining from an authorization table stored by the application server whether any other voice assistant devices have been authorized for communication with the application server;
in response to a determination that one or more other voice assistant devices have been authorized for communication with the application server:
causing the first voice assistant device to generate a prompt for input whether to transfer the first secure communication session to one of the one or more other voice assistant devices that have been authorized for communication with the application server;
in response to input to transfer the first secure communication session to one of the one or more other voice assistant devices that have been authorized for communication with the application server:
initiating a second secure communication session between a second voice assistant device and the application server; and
terminating the first secure communication session between the first voice assistant device and the application server in response to successful initiation of the second secure communication session between the second voice assistant device and the application server.
