# US 11,908,459 B2 · Detection of potential exfiltration of audio data from digital assistant applications

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,908,459 B2
- Document ID: US11908459B2
- Status: Granted
- Kind: B2
- Application number: 17/321,038
- Filed: 2021-05-14
- Published: 2021-09-02
- Granted: 2024-02-20
- Earliest claimed: 2018-09-14
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Yan Huang; Nikhil Rao
- Web page: https://patentgrep.com/patent/US11908459B2

## CPC

- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G10L 15/18: using natural language modelling
- G06F 21/60: Protecting data
- G06F 21/6245: {Protecting personal data, e.g. for financial or medical purposes}
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L 25/54: for retrieval
- G10L 25/78: Detection of presence or absence of voice signals (switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10)
- G06F 21/554: {involving event detection and direct action}
- G06F 21/556: {involving covert channels, i.e. data leakage between processes (inhibiting the analysis of circuitry or operation with measures against power attack G06F21/755)}
- G10L 2015/223: {Execution procedure of a spoken command}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

The present disclosure is generally related to a data processing system to detect potential exfiltration of audio data by agent applications can include a data processing system. The data processing system can identify, from an I/O record, an input received from the digital assistant application via a microphone of a client device, an output received from the agent application after the input, and a microphone status for the microphone. The data processing system can determine that the output is terminal based on the input and the output. The data processing system can identify the microphone status as in the enabled state subsequent to the input. The data processing system can determine that the agent application is unauthorized to access audio data acquired via the microphone of the client device based on determining that the output is terminal and identifying the microphone status as enabled.

## Claims

### Claim 1

A method implemented by one or more processors, the method comprising:
receiving a training dataset including sample input strings and sample output strings that are each labelled as a terminal statement, an anticipatory statement, or an expectative statement;
applying each of the sample input strings and each of the sample output strings onto inputs of an I/O exchange model;
obtaining confidence metrics calculated using the I/O exchange model for each of the sample input strings and each of the sample output strings;
adjusting weights of the I/O exchange model based on a comparison of the confidence metrics and the labels for the sample input strings and the sample output strings;
identifying an input received from a digital assistant application via a microphone of a client device and an output received from an agent application that interfaces with the digital assistant application subsequent to the input;
determining, using the I/O exchange model, that the output is terminal to the input based on a first set of strings included in the input and a second set of strings included in the output; and
performing an action on at least one of the agent application and the digital assistant application based on the determining that the output is terminal to the input.

### Claim 2 (depends on 1)

The method according to claim 1, wherein:
the training dataset further includes sample pairs of input strings and output strings, and in each of the sample pairs, the input string or the output string is labelled as the terminal statement, the anticipatory statement, or the expectative statement;
the applying further comprises applying each of the sample pairs of input strings and output strings onto the inputs of the I/O exchange model;
the obtaining confidence metrics further comprises obtaining confidence metrics calculated using the I/O exchange model for each of the sample pairs of input strings and output strings; and
the adjusting weights is further based on a comparison of the confidence metrics and the labels for the sample pairs of input strings and output strings.

### Claim 3 (depends on 1)

The method according to claim 1, wherein the training dataset includes a device modality of a client device associated with the sample input strings and the sample output strings.

### Claim 4 (depends on 1)

The method according to claim 1, further comprising repeating the applying, the obtaining confidence metrics, and the adjusting weights until convergence of the I/O exchange model.

### Claim 5 (depends on 1)

The method according to claim 1, wherein performing the action on at least one of the agent application and the digital assistant application is further based on a microphone status of the microphone subsequent to the input.

### Claim 6 (depends on 5)

The method according to claim 5, wherein the microphone status indicates one of an enabled state and a disabled state of the microphone subsequent to the input.

### Claim 7

A system comprising:
a processor, a computer-readable memory, one or more computer-readable storage media, and program instructions collectively stored on the one or more computer-readable storage media, the program instructions executable to:
receive a training dataset including sample input strings and sample output strings that are each labelled as a terminal statement, an anticipatory statement, or an expectative statement;
apply each of the sample input strings and each of the sample output strings onto inputs of an I/O exchange model;
obtain confidence metrics calculated using the I/O exchange model for each of the sample input strings and each of the sample output strings;
adjust weights of the I/O exchange model based on a comparison of the confidence metrics and the labels for the sample input strings and the sample output strings;
identify an input received from a digital assistant application via a microphone of a client device and an output received from an agent application that interfaces with the digital assistant application subsequent to the input;
determine, using the I/O exchange model, that the output is terminal to the input based on a first set of strings included in the input and a second set of strings included in the output; and
perform an action on at least one of the agent application and the digital assistant application based on the determining that the output is terminal to the input.

### Claim 8 (depends on 7)

The system according to claim 7, wherein:
the training dataset further includes sample pairs of input strings and output strings, and in each of the sample pairs, the input string or the output string is labelled as the terminal statement, the anticipatory statement, or the expectative statement;
the applying further comprises applying each of the sample pairs of input strings and output strings onto the inputs of the I/O exchange model;
the obtaining confidence metrics further comprises obtaining confidence metrics calculated using the I/O exchange model for each of the sample pairs of input strings and output strings; and
the adjusting weights is further based on a comparison of the confidence metrics and the labels for the sample pairs of input strings and output strings.

### Claim 9 (depends on 7)

The system according to claim 7, wherein the training dataset includes a device modality of a client device associated with the sample input strings and the sample output strings.

### Claim 10 (depends on 7)

The system according to claim 7, wherein the program instructions are further executable to repeat the applying, the obtaining confidence metrics, and the adjusting weights until convergence of the I/O exchange model.

### Claim 11 (depends on 7)

The system according to claim 7, wherein performing the action on at least one of the agent application and the digital assistant application is further based on a microphone status of the microphone subsequent to the input.

### Claim 12

A method implemented by one or more processors, the method comprising:
receiving a training dataset including sample input strings and sample output strings that are each labelled as a terminal statement, an anticipatory statement, or an expectative statement;
applying each of the sample input strings and each of the sample output strings onto inputs of an I/O exchange model;
obtaining confidence metrics calculated using the I/O exchange model for each of the sample input strings and each of the sample output strings; and
adjusting weights of the I/O exchange model based on a comparison of the confidence metrics and the labels for the sample input strings and the sample output strings.

### Claim 13 (depends on 12)

The method according to claim 12, wherein:
the training dataset further includes sample pairs of input strings and output strings, and in each of the sample pairs, the input string or the output string is labelled as the terminal statement, the anticipatory statement, or the expectative statement;
the applying further comprises applying each of the sample pairs of input strings and output strings onto the inputs of the I/O exchange model;
the obtaining confidence metrics further comprises obtaining confidence metrics calculated using the I/O exchange model for each of the sample pairs of input strings and output strings; and
the adjusting weights is further based on a comparison of the confidence metrics and the labels for the sample pairs of input strings and output strings.

### Claim 14 (depends on 12)

The method according to claim 12, wherein the training dataset includes a device modality of a client device associated with the sample input strings and the sample output strings.

### Claim 15 (depends on 12)

The method according to claim 12, further comprising repeating the applying, the obtaining confidence metrics, and the adjusting weights until convergence of the I/O exchange model.