# US 2015/0149174 A1 · Differential acoustic model representation and linear transform-based adaptation for efficient user profile update techniques in automatic speech recognition

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2015/0149174 A1
- Document ID: US20150149174A1
- Status: Published application
- Kind: A1
- Application number: 14/399,867
- Filed: 2012-05-08
- Published: 2015-05-28
- Earliest claimed: 2012-05-08
- Assignees: Nuance Communications, Inc.
- Inventors: Christian Gollan; Daniel Willett
- Web page: https://patentgrep.com/patent/US20150149174A1

## CPC

- G10L 17/06: Decision making techniques; Pattern matching strategies
- G10L 15/07: to the speaker
- G10L 15/285: {Memory allocation or algorithm optimisation to reduce hardware requirements}
- G10L 15/30: Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
- G10L 2015/227: {of the speaker; Human-factor methodology}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

A computer-implemented method is described for speaker adaptation in automatic speech recognition. Speech recognition data from a particular speaker is used for adaptation of an initial speech recognition acoustic model to produce a speaker adapted acoustic model. A speaker dependent differential acoustic model is determined that represents differences between the initial speech recognition acoustic model and the speaker adapted acoustic model. In addition, an approach is also disclosed to estimate speaker-specific feature or model transforms over multiple sessions. This is achieved by updating the previously estimated transform using only adaptation statistics of the current session.

## Claims

### Claim 1

A method comprising:
adapting, by a computing device, an initial speech recognition acoustic model to produce a speaker adapted acoustic model using speech recognition data from a particular speaker;
developing, by the computing device, a speaker differential acoustic model representing one or more differences between the initial speech recognition acoustic model and the speaker adapted acoustic model; and
storing the speaker differential acoustic model for subsequent speech recognition with the particular speaker.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
retrieving the stored speaker differential acoustic model; and
processing the speaker differential acoustic model and the initial speech recognition acoustic model to derive the speaker adapted acoustic model.

### Claim 3 (depends on 2)

The method of claim 2, further comprising:
using the derived speaker adapted acoustic model to perform speech recognition for the particular speaker.

### Claim 4 (depends on 1)

The method of claim 1, wherein the storing the speaker differential acoustic model includes selecting a limited subset of parameter differences for storing to control size of the speaker differential acoustic model.

### Claim at position 5

5-8. (canceled)

### Claim 9

A method comprising:
loading a user profile for a particular speaker including an initial user feature transform representing speaker adapted speech recognition acoustic models;
performing speech recognition, by a computing device, for a session of speech utterances from the particular speaker using the initial user feature transform and a plurality of speaker independent speech recognition acoustic models;
determining a session update transform for a linear transform-based speaker adaptation of the initial user feature transform based on speech recognition data from the session;
producing an updated user feature transform by combining the initial user feature transform and the session update transform; and
storing in the user profile the updated user feature transform for subsequent speech recognition with the particular speaker.

### Claim 10 (depends on 9)

The method of claim 9, wherein producing the updated user feature transform by combining the initial user feature transform and the session update transform is performed without a bias vector.

### Claim 11 (depends on 9)

The method of claim 9, wherein producing the updated user feature transform by combining the initial user feature transform and the session update transform is performed with a bias vector.

### Claim 12 (depends on 11)

The method of claim 11, wherein the bias vector is an acoustic observation vector for constrained maximum likelihood linear regression (CMLLR) adaptation.

### Claim 13 (depends on 11)

The method of claim 11, wherein the bias vector is a component mean vector for maximum likelihood linear regression (MLLR) adaptation.

### Claim 14 (depends on 9)

The method of claim 9, wherein the linear transform-based speaker adaptation uses relevance smoothing of speech recognition data from the session.

### Claim 15

A non-transitory computer-readable medium storing computer-readable instructions that, when executed by a processor, cause a device to:
load a user profile for a particular speaker including an initial user feature transform representing speaker adapted speech recognition acoustic models;
perform speech recognition for a session of speech utterances from the particular speaker using the initial user feature transform and a plurality of speaker independent speech recognition acoustic models;
determine a session update transform for a linear transform-based speaker adaptation of the initial user feature transform based on speech recognition data from the session;
produce an updated user feature transform by combining the initial user feature transform and the session update transform; and
store in the user profile the updated user feature transform for subsequent speech recognition with the particular speaker.

### Claim 16 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein producing the updated user feature transform by combining the initial user feature transform and the session update transform is performed without a bias vector.

### Claim 17 (depends on 16)

The non-transitory computer-readable medium of claim 16, wherein producing the updated user feature transform by combining the initial user feature transform and the session update transform is performed with a bias vector.

### Claim 18 (depends on 17)

The non-transitory computer-readable medium of claim 17, wherein the bias vector is an acoustic observation vector for constrained maximum likelihood linear regression (CMLLR) adaptation.

### Claim 19 (depends on 17)

The non-transitory computer-readable medium of claim 17, wherein the bias vector is a component mean vector for maximum likelihood linear regression (MLLR) adaptation.

### Claim 20 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein the linear transform-based speaker adaptation uses relevance smoothing of speech recognition data from the session.

### Claim 21 (depends on 4)

The method of claim 4, wherein the limited subset of parameter differences comprises most important parameter differences of the parameter differences.

### Claim 22 (depends on 1)

The method of claim 1, further comprising:
ranking the one or more differences determined using a scoring function;
keeping a subset of the one or more differences, the subset selected based on the rank of the one or more differences;
optimizing different quantization ranges for parameter subsets of the speaker adapted acoustic model using the scoring function;
finding storage-efficient Gaussian component clustering of the speaker differential acoustic model using the scoring function; and
using the subset of the one or more differences, the different quantization ranges, and the storage-efficient Gaussian component clustering to minimize an acoustic model difference measure of the speaker adapted acoustic model against the initial speech recognition acoustic model.

### Claim 23 (depends on 3)

The method of claim 3, wherein performing the speech recognition comprises:
producing a representative recognition output of the speech recognition data from the particular speaker, wherein the representative recognition output is constrained by a recognition language model and search algorithm; and
comparing the representative recognition output to the derived speaker adapted acoustic model.

### Claim 24 (depends on 1)

The method of claim 1, further comprising:
adapting the speaker adapted acoustic model based on specific technical characteristics of a device used by the particular speaker.
