# US 2017/0256264 A1 · System and Method for Performing Dual Mode Speech Recognition

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0256264 A1
- Document ID: US20170256264A1
- Status: Published application
- Kind: A1
- Application number: 15/603,257
- Filed: 2017-05-23
- Published: 2017-09-07
- Earliest claimed: 2011-11-18
- Assignees: SoundHound, Inc.
- Applicants: SoundHound, Inc.
- Inventors: Timothy Stonehocker; Keyvan Mohajer; Bernard Mont-Reynaud
- Web page: https://patentgrep.com/patent/US20170256264A1

## CPC

- G10L 15/30: Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
- G10L 15/04: Segmentation; Word boundary detection
- G10L 15/063: {Training}
- G10L 15/08: Speech classification or search
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- G10L 17/06: Decision making techniques; Pattern matching strategies
- G10L 15/34: Adaptation of a single recogniser for parallel processing, e.g. by use of multiple processors or cloud computing
- G10L 2015/0635: {updating or merging of old and new templates; Mean values; Weighting}
- G10L 2015/081: {Search algorithms, e.g. Baum-Welch or Viterbi}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

A system and method is presented for performing dual mode speech recognition, employing a local recognition module on a mobile device and a remote recognition engine on a server device. The system accepts a spoken query from a user, and both the local recognition module and the remote recognition engine perform speech recognition operations on the query, returning a transcription and confidence score, subject to a latency cutoff time. If both sources successfully transcribe the query, then the system accepts the result having the higher confidence score. If only one source succeeds, then that result is accepted. In either case, if the remote recognition engine does succeed in transcribing the query, then a client vocabulary is updated if the remote system result includes information not present in the client vocabulary.

## Claims

### Claim 1

A method comprising:
transmitting a spoken query to a remote recognition system;
obtaining a server transcription from the remote recognition system; and
responsive to determining that the server transcription contains a word that is missing from a local vocabulary, issuing a request to the server to send a description of the missing word.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
receiving the description of the missing word; and
updating the local vocabulary with the description of the missing word.

### Claim 3 (depends on 2)

The method of claim 2, further comprising:
receiving descriptions of words related to the missing word; and
updating the local vocabulary with the descriptions of the related words.

### Claim 4 (depends on 3)

The method of claim 3 wherein the missing word and the related words are related to the same topic.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
performing local speech recognition on the spoken query, using the local vocabulary;
obtaining a server recognition score; and
comparing the server recognition score to a local speech recognition score,
wherein issuing the request to the server to send a description of the missing word depends on the outcome of the score comparison.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
responsive to available memory resources for client vocabulary data being about to run out, performing a garbage collection operation.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
combining frequency and recency of use to determine a word frequency amortized over time; and
choosing a priority of a non-permanent word using the word frequency amortized over time.

### Claim 8 (depends on 1)

The method of claim 1, wherein the description includes a phonetic lattice.

### Claim 9

A method comprising:
receiving a request from a client device to send a description of a missing word;
determining a topic to which the missing word relates; and
sending to the client a set of words that relate to the topic and descriptions of the words.

### Claim 10

A non-transitory computer readable medium comprising code that, when executed by one or more processors, causes the one or more processors to:
transmit a spoken query to a remote recognition system;
obtain a server transcription from the remote recognition system; and
responsive to determining that the server transcription contains a word that is missing from a local vocabulary, issue a request to the server to send a description of the missing word.

### Claim 11 (depends on 10)

The non-transitory computer readable medium of claim 10, further comprising code that, when executed by one or more processors, causes the one or more processors to:
receive the description of the missing word; and
update the local vocabulary with the description of the missing word.

### Claim 12 (depends on 11)

The non-transitory computer readable medium of claim 11, further comprising code that, when executed by one or more processors, causes the one or more processors to:
receive descriptions of words related to the missing word; and
update the local vocabulary with the descriptions of the related words.

### Claim 13 (depends on 12)

The non-transitory computer readable medium of claim 12, wherein the missing word and the related words are related to the same topic.

### Claim 14 (depends on 10)

The non-transitory computer readable medium of claim 10, further comprising code that, when executed by one or more processors, causes the one or more processors to:
perform local speech recognition on the spoken query, using the local vocabulary;
obtain a server recognition score; and
compare the server recognition score to a local speech recognition score,
wherein issuing the request to the server to send a description of the missing word depends on the outcome of the score comparison.

### Claim 15 (depends on 10)

The non-transitory computer readable medium of claim 10, further comprising code that, when executed by one or more processors, causes the one or more processors to:
responsive to available memory resources for client vocabulary data being about to run out, perform a garbage collection operation.

### Claim 16 (depends on 10)

The non-transitory computer readable medium of claim 10, further comprising code that, when executed by one or more processors, causes the one or more processors to:
combine frequency and recency of use to determine a word frequency amortized over time; and
choose a priority of a non-permanent word using the word frequency amortized over time.

### Claim 17 (depends on 10)

The non-transitory computer readable medium of claim 10, wherein the description includes a phonetic lattice.
