Back to results
US 11,238,227 B2Granted
Word lattice augmentation for automatic speech recognition
AssigneeGoogle Inc.
InventorsLeonid Velikovich, Petar Aleksic, Pedro Moreno
Life of this patent
Estimating the term…
Abstract
Speech processing techniques are disclosed that enable determining a text representation of named entities in captured audio data. Various implementations include determining the location of a carrier phrase in a word lattice representation of the captured audio data, where the carrier phrase provides an indication of a named entity. Additional or alternative implementations include matching a candidate named entity with the portion of the word lattice, and augmenting the word lattice with the matched candidate named entity.
Claims14
A method implemented by one or more processors, the method comprising:
- generating a text representation of captured audio data using an automatic speech recognition engine, wherein generating the text representation of the captured audio data comprises:
- generating a word lattice corresponding to the captured audio data, wherein the word lattice is an acyclic graph representing candidate recognitions of the captured audio data;
- determining, based on a location of a carrier phrase in the word lattice, an augmentation location in the word lattice;
- selecting, from a set of candidate contextually relevant named entities, a contextually relevant named entity to insert at the augmentation location of the word lattice,
- wherein the contextually relevant named entity is an additional candidate recognition of a portion of the captured audio data, and
- wherein selecting, from the set of candidate contextually relevant named entities, the contextually relevant named entity to insert at the augmentation location of the word lattice comprises:
- determining a set of candidate phoneme sequences corresponding to the word lattice at the augmentation location, wherein determining the set of candidate phoneme sequences corresponding to the word lattice at the augmentation location comprises:
- for each phoneme in the set of candidate phoneme sequences:
- determining whether the phoneme has one or more acoustically similar phonemes;
- and
- in response to determining the phoneme has one or more acoustically similar phonemes, inserting the one or more acoustically similar phonemes as one or more additional sequence paths in the set of candidate phoneme sequences;
- and
- selecting the contextually relevant named entity by matching a phoneme representation of a candidate contextually relevant named entity in the set of candidate contextually relevant named entities with a candidate phoneme sequence in the set of candidate phoneme sequences;
- augmenting the word lattice by inserting the selected contextually relevant named entity at the augmentation location;
- and
- processing the augmented word lattice to determine the text representation of the captured audio data.
13 more elements
A computing device comprising:
- memory storing instructions;
- one or more processors executing the instructions to:
- generate a text representation of captured audio data using an automatic speech recognition engine, wherein in generating the text representation of the captured audio data one or more of the processors are to:
- generate a word lattice corresponding to the captured audio data, wherein the word lattice is an acyclic graph representing candidate recognitions of the captured audio data;
- determine, based on a location of a carrier phrase in the word lattice, an augmentation location in the word lattice;
- select, from a set of candidate contextually relevant named entities, a contextually relevant named entity to insert at the augmentation location of the word lattice, wherein the contextually relevant named entity is an additional candidate recognition of a portion of the captured audio data, and wherein selecting, from the set of candidate contextually relevant named entities, the contextually relevant named entity to insert at the augmentation location of the word lattice comprises:
- determining a set of candidate phoneme sequences corresponding to the word lattice at the augmentation location, wherein determining the set of candidate phoneme sequences corresponding to the word lattice at the augmentation location comprises:
- for each candidate phoneme sequence in the set of candidate phoneme sequences:
- inserting one or more edited phoneme sequences as additional sequence paths in the set of candidate phoneme sequences,
- where each edited phoneme sequence is a defined distance away from the candidate phoneme sequence;
- and
- selecting the contextually relevant named entity by matching a phoneme representation of a candidate contextually relevant named entity in the set of candidate contextually relevant named entities with a candidate phoneme sequence in the set of candidate phoneme sequences;
- augment the word lattice by inserting the selected contextually relevant named entity at the augmentation location;
- and
- process the augmented word lattice to determine the text representation of the captured audio data.
12 more elements
Dependencies are inferred from the claim text.