# US 2020/0286473 A1 · Systems and method to resolve audio-based requests in a networked environment

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0286473 A1
- Document ID: US20200286473A1
- Status: Published application
- Kind: A1
- Application number: 16/883,690
- Filed: 2020-05-26
- Published: 2020-09-10
- Earliest claimed: 2018-04-16
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Pedro Gonnet ANDERS; Victor CARBUNE; Daniel KEYSERS; Thomas DESELAERS; Sandro FEUZ
- Web page: https://patentgrep.com/patent/US20200286473A1

## CPC

- G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning}
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G06F 16/90332: {Natural language query formulation or dialogue systems}
- G06F 16/9035: Filtering based on additional data, e.g. user or group profiles
- G06F 40/35: Discourse or dialogue representation
- G06F 40/56: Natural language generation
- G10L 15/08: Speech classification or search
- G10L 15/19: Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L 15/18: using natural language modelling
- G10L 2015/223: {Execution procedure of a spoken command}
- G10L 2015/227: {of the speaker; Human-factor methodology}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Techniques are described herein for enabling an automated assistant to adjust its behavior depending on a detected vocabulary level or other vocal characteristics of an input utterance provided to an automated assistant. The estimated vocabulary level or other vocal characteristics may be used to influence various aspects of a data processing pipeline employed by the automated assistant. In some implementations, one or more tolerance thresholds associated with, for example, grammatical tolerances or vocabulary tolerances, may be adjusted based on the estimated vocabulary level or vocal characteristics of the input utterance.

## Claims

### Claim at position 1

-20. (canceled)

### Claim 21

A system to resolve requests in audio-based networked environments, comprising:
a data processing system having one or more processors coupled with memory to:
parse an input audio acquired via a sensor at a client device to determine a characteristic, the characteristic including at least one of a vocal characteristic, a grammar level, or a vocabulary level;
select, from a plurality of interaction models, an interaction model based on the characteristic;
identify a digital component based on one or more keywords identified from the input audio in accordance with the interaction model; and
provide, in response to the input audio, the digital component to present via the client device.

### Claim 22 (depends on 21)

The system of claim 21, comprising the data processing system to:
identify a trigger to disambiguate the parsing of the input audio; and
select, responsive to the identification of the trigger to disambiguate, the interaction model from the plurality of interaction models to process the input audio.

### Claim 23 (depends on 21)

The system of claim 21, comprising the data processing system to:
identify an account profile associated with the client device via which the input audio is acquired; and
select, from the plurality of interaction models, the interaction model based on the account profile.

### Claim 24 (depends on 21)

The system of claim 21, comprising the data processing system to:
parse, using a natural language processor, the input audio to identify a plurality of candidate intents;
select, from the plurality of candidate intents, an intent based on the one or more keywords of the input audio identified in accordance with the interaction model; and
identify the digital component based on the intent identified from the plurality of candidate intents.

### Claim 25 (depends on 21)

The system of claim 21, comprising the data processing system to:
identify an intent based on at least one of the one or more keywords identified from the input audio in accordance with the interaction model, the intent defined for a third-party application interfacing with a digital assistant application executing on the client device; and
identify the digital component based on the intent defined for the third-party application.

### Claim 26 (depends on 21)

The system of claim 21, comprising the data processing system to:
identify an intent based on a mapping defined by the interaction model between at least one of the one or more keywords to one of a plurality of intents; and
identify the digital component based on the intent defined by the mapping.

### Claim 27 (depends on 21)

The system of claim 21, comprising the data processing system to:
identify a first keyword from the input audio using the interaction model; and
determine, using an entity data structure defining relationships among a plurality of keywords, a second keyword associated with first keyword to identify the digital component.

### Claim 28 (depends on 21)

The system of claim 21, comprising the data processing system to:
select the interaction model from the plurality of interaction models, each of the plurality of interaction models having a tolerance level for at least one of a pronunciation, a grammar, and a vocabulary in processing the input audio; and
modify a first keyword parsed from the input audio to a second keyword in accordance with the tolerance level of the interaction model to identify the digital component.

### Claim 29 (depends on 21)

The system of claim 21, comprising the data processing system to:
select, from a plurality of voice synthesis models, a voice synthesis model based on the characteristic determined from the input audio; and
provide the digital component to present via the client device using the voice synthesis model.

### Claim 30 (depends on 21)

The system of claim 21, comprising the data processing system to
identify a second digital component associated with the digital component based on the one or more keywords identified from the input audio in accordance with the interaction model; and
provide, in response to the input audio, the digital component and the second digital component to present via the client device.

### Claim 31

A method of resolving requests in audio-based networked environments, comprising:
parsing, by a data processing system, an input audio acquired via a sensor at a client device to determine a characteristic, the characteristic including at least one of a vocal characteristic, a grammar level, or a vocabulary level;
selecting, by the data processing system, from a plurality of interaction models, an interaction model based on the characteristic;
identifying, by the data processing system, a digital component based on one or more keywords identified from the input audio in accordance with the interaction model; and
providing, by the data processing system, in response to the input audio, the digital component to present via the client device.

### Claim 32 (depends on 31)

The method of claim 31, comprising:
identifying, by the data processing system, a trigger to disambiguate the parsing of the input audio; and
selecting, by the data processing system, responsive to the identification of the trigger to disambiguate, the interaction model from the plurality of interaction models to process the input audio.

### Claim 33 (depends on 31)

The method of claim 31, comprising:
identifying, by the data processing system, an account profile associated with the client device via which the input audio is acquired; and
selecting, by the data processing system, from the plurality of interaction models, the interaction model based on the account profile.

### Claim 34 (depends on 31)

The method of claim 31, comprising:
parsing, by the data processing system, using a natural language processor, the input audio to identify a plurality of candidate intents;
selecting, by the data processing system, from the plurality of candidate intents, an intent based on the one or more keywords of the input audio identified in accordance with the interaction model; and
identifying, by the data processing system, the digital component based on the intent identified from the plurality of candidate intents.

### Claim 35 (depends on 31)

The method of claim 31, comprising:
identifying, by the data processing system, an intent based on at least one of the one or more keywords identified from the input audio in accordance with the interaction model, the intent defined for a third-party application interfacing with a digital assistant application executing on the client device; and
identifying, by the data processing system, the digital component based on the intent defined for the third-party application.

### Claim 36 (depends on 31)

The method of claim 31, comprising:
identifying, by the data processing system, an intent based on a mapping defined by the interaction model between at least one of the one or more keywords to one of a plurality of intents; and
identifying, by the data processing system, the digital component based on the intent defined by the mapping.

### Claim 37 (depends on 31)

The method of claim 31, comprising:
identifying, by the data processing system, a first keyword from the input audio using the interaction model; and
determining, by the data processing system, using an entity data structure defining relationships among a plurality of keywords, a second keyword associated with first keyword to identify the digital component.

### Claim 38 (depends on 31)

The method of claim 31, comprising:
selecting, by the data processing system, the interaction model from the plurality of interaction models, each of the plurality of interaction models having a tolerance level for at least one of a pronunciation, a grammar, and a vocabulary in processing the input audio; and
modifying, by the data processing system, a first keyword parsed from the input audio to a second keyword in accordance with the tolerance level of the interaction model to identify the digital component.

### Claim 39 (depends on 31)

The method of claim 31, comprising:
selecting, by the data processing system, from a plurality of voice synthesis models, a voice synthesis model based on the characteristic determined from the input audio; and
providing, by the data processing system, the digital component to present via the client device using the voice synthesis model.

### Claim 40 (depends on 31)

The method of claim 31, comprising:
identifying, by the data processing system, a second digital component associated with the digital component based on the one or more keywords identified from the input audio in accordance with the interaction model; and
providing, by the data processing system, in response to the input audio, the digital component and the second digital component to present via the client device.
