# US 2012/0259631 A1 · Speech and Noise Models for Speech Recognition

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2012/0259631 A1
- Document ID: US20120259631A1
- Status: Published application
- Kind: A1
- Application number: 13/530,614
- Filed: 2012-06-22
- Published: 2012-10-11
- Earliest claimed: 2010-06-14
- Assignees: Google Inc.
- Inventors: Matthew I. Lloyd; Trausti T. Kristjansson
- Web page: https://patentgrep.com/patent/US20120259631A1

## CPC

- G10L 15/20: Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech (G10L21/02 takes precedence)
- G10L 21/0208: Noise filtering
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

An audio signal generated by a device based on audio input from a user may be received. The audio signal may include at least a user audio portion that corresponds to one or more user utterances recorded by the device. A user speech model associated with the user may be accessed and a determination may be made background audio in the audio signal is below a defined threshold. In response to determining that the background audio in the audio signal is below the defined threshold, the accessed user speech model may be adapted based on the audio signal to generate an adapted user speech model that models speech characteristics of the user. Noise compensation may be performed on the received audio signal using the adapted user speech model to generate a filtered audio signal with reduced background audio compared to the received audio signal.

## Claims

### Claim 1

A system comprising:
one or more processing devices; and
one or more storage devices storing instructions that, when executed by the one or more processing devices, cause the system to:
receive an audio signal generated by a device based on audio input from a user, the audio signal including at least background audio and one or more user utterances recorded by the device;
determine a location of the user when the one or more utterances are recorded;
select a noise model from a plurality of noise models;
adapt the selected noise model based on the received audio signal to generate an adapted noise model that models characteristics of background audio surrounding the user at the location; and
store the adapted noise model as a noise model for the user in association with the determined location such that the adapted noise model is used for speech recognition when utterances of the user are recorded at the determined location.

### Claim 2 (depends on 1)

The system of claim 1, wherein the instructions further comprise instructions that, when executed, cause the system to generate a filtered audio signal with reduced background audio compared to the received audio signal using the selected noise model.

### Claim 3 (depends on 1)

The system of claim 1, wherein:
to determine the location of the user when the one or more utterances are recorded, the instructions, when executed, cause the system to determine a geographical location of the user when the one or more utterances are recorded; and
to store the adapted noise model as a noise model for the user for audio input recorded at the location, the instructions, when executed, cause the system to store the adapted noise model as a noise model for the user for audio input recorded at the geographical location.

### Claim 4 (depends on 1)

The system of claim 1, wherein:
to determine the location of the user when the one or more utterances are recorded, the instructions, when executed, cause the system to determine a type of location where the user is located when the one or more utterances are recorded; and
to store the adapted noise model as a noise model for the user for audio input recorded at the location, the instructions, when executed, cause the system to store the adapted noise model as a noise model for the user for audio input recorded at locations of the same type as the determined type of location.

### Claim 5 (depends on 1)

The system of claim 1, wherein, to select the noise model, the instructions, when executed, cause the system to select the noise model based on the determined location.

### Claim 6 (depends on 5)

The system of claim 5, wherein, to select the noise model based on the determined location, the instructions, when executed, cause the system to select a noise model associated with the user that was previously adapted to model characteristics of background audio at the determined location.

### Claim 7 (depends on 6)

The system of claim 6, wherein, to select the noise model based on the determined location, the instructions, when executed, cause the system to select the noise model from among a plurality of noise models that were each previously adapted to model characteristics of background audio at a corresponding location based on audio input from the user.

### Claim 8 (depends on 5)

The system of claim 5, wherein, to select the noise model based on the determined location, the instructions, when executed, cause the system to select the noise model from among a plurality of noise models that are each developed to model characteristics of background audio at a corresponding location based on audio input that is recorded at the corresponding location by other users.

### Claim 9 (depends on 8)

The system of claim 8, wherein the audio input that is recorded at the determined location by other users comprises voice queries submitted by the other users.

### Claim 10 (depends on 1)

The system of claim 1, wherein, to store the adapted noise model as a noise model for the user, the instructions, when executed, cause the system to store the adapted noise model as one of a plurality of location-specific noise models adapted for the user, each of the location-specific noise models modeling characteristics of background audio at a different location.

### Claim 11 (depends on 1)

The system of claim 1, wherein the instructions further comprise instructions that, when executed, cause the system to:
receive a second audio signal generated by the device based on second audio input from the user, the audio signal including at least second background audio and one or more second user utterances recorded by the device;
determine that, when the one or more second utterances are recorded, the user is at the determined location where the one or more first utterances were recorded; and
perform noise compensation on the second audio signal using the adapted noise model in response to determining that, when the one or more second utterances are recorded, the user is at the determined location where the one or more first utterances were recorded.

### Claim 12 (depends on 1)

The system of claim 1, wherein the instructions further comprise instructions that, when executed, cause the system to:
receive a second audio signal generated by the device based on second audio input from the user, the audio signal including at least second background audio and one or more second user utterances recorded by the device;
determine a second location of the user when the one or more second utterances are recorded, the second location being different from the location of the user when the one or more first utterances are recorded;
select a second noise model based on the second location, from among multiple noise models associated with the user, each of the multiple noise models associated with the user being adapted based on audio input from the user and modeling characteristics of background noise at a different location; and
perform noise compensation on the second audio signal using the second noise model.

### Claim 13 (depends on 1)

The system of claim 1, wherein the instructions further comprise instructions that, when executed, cause the system to:
access a user speech model associated with the user that models speech characteristics of the user; and
perform noise compensation on the received audio signal using the user speech model and the selected noise model.

### Claim 14 (depends on 13)

The system of claim 13, wherein:
the instructions comprise instructions that, when executed, cause the system to determine that the background audio in the audio signal is above a defined threshold; and
to adapt the selected noise model, the instructions, when executed, cause the system to adapt the selected noise model in response to determining that the background audio in the audio signal is above the defined threshold.

### Claim 15 (depends on 1)

The system of claim 1, wherein the selected noise model comprises a surrogate noise model that has not been adapted to model characteristics of background audio surrounding the user.

### Claim 16 (depends on 15)

The system of claim 15, wherein, to select the surrogate noise model, the instructions include instructions that, when executed by the one or more processing devices, cause the system to:
receive an initial audio signal that includes at least an initial user audio portion that corresponds to one or more user utterances recorded by the device;
determine similarity metrics between multiple surrogate noise models and an expected noise model for the user determined based on the initial audio signal; and
select the surrogate noise model, from among multiple surrogate noise models, based on the similarity metrics.

### Claim 17 (depends on 16)

The system of claim 16, wherein each of the multiple surrogate noise models model characteristics of background audio in a particular location.

### Claim 18 (depends on 16)

The system of claim 16, wherein each of the multiple surrogate noise models model characteristics of background audio in a particular kind of environmental condition.

### Claim 19 (depends on 15)

The system of claim 15, wherein, to select the surrogate noise model, the instructions include instructions that, when executed by the one or more processing devices, cause the system to:
receive context information about the device; and
select the surrogate noise model, from among the multiple surrogate noise models, based on the context information.

### Claim 20

A computer-implemented method comprising:
receiving an audio signal generated by a device based on audio input from a user, the audio signal including at least background audio and one or more user utterances recorded by the device;
determining a location of the user when the one or more utterances are recorded;
selecting a noise model from a plurality of noise models;
adapting the selected noise model based on the received audio signal to generate an adapted noise model that models characteristics of background audio surrounding the user at the location; and
storing the adapted noise model as a noise model for the user in association with the determined location such that the adapted noise model is used for speech recognition when utterances of the user are recorded at the determined location.

### Claim 21

A computer storage medium encoded with a computer program, the program comprising instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:
receiving an audio signal generated by a device based on audio input from a user, the audio signal including at least background audio and one or more user utterances recorded by the device;
determining a location of the user when the one or more utterances are recorded;
selecting a noise model from a plurality of noise models;
adapting the selected noise model based on the received audio signal to generate an adapted noise model that models characteristics of background audio surrounding the user at the location; and
storing the adapted noise model as a noise model for the user in association with the determined location such that the adapted noise model is used for speech recognition when utterances of the user are recorded at the determined location.
