# US 2011/0238415 A1 · Hybrid Speech Recognition

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2011/0238415 A1
- Document ID: US20110238415A1
- Status: Published application
- Kind: A1
- Application number: 12/890,280
- Filed: 2010-09-24
- Published: 2011-09-29
- Earliest claimed: 2008-08-29
- Assignees: Multimodal Technologies, LLC
- Inventors: Detlef Koll
- Web page: https://patentgrep.com/patent/US20110238415A1

## CPC

- G10L 15/30: Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
- G10L 15/32: Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

A hybrid speech recognition system uses a client-side speech recognition engine and a server-side speech recognition engine to produce speech recognition results for the same speech. An arbitration engine produces speech recognition output based on one or both of the client-side and server-side speech recognition results.

## Claims

### Claim 1

A computer-implemented method performed by a client device, the method comprising:
(A) receiving a request from a requester to apply automatic speech recognition to an audio signal;
(B) providing the audio signal to a first automatic speech recognition engine in the client device;
(C) receiving first speech recognition results from the first automatic speech recognition engine;
(D) determining whether a second automatic speech recognition engine, in a server device, is accessible to the client device;
(E) if the second automatic speech recognition engine is determined not to be accessible to the client device, then providing the first speech recognition results to the requester in response to the request.

### Claim 2 (depends on 1)

The method of claim 1, wherein the requester comprises a machine coupled to the client device.

### Claim 3 (depends on 1)

The method of claim 1, wherein the requester comprises software executing on the client device.

### Claim 4

An apparatus comprising:
means for receiving a request from a requester to apply automatic speech recognition to an audio signal;
means for providing the audio signal to a first automatic speech recognition engine in the client device;
means for receiving first speech recognition results from the first automatic speech recognition engine;
means for determining whether a second automatic speech recognition engine, in a server device, is accessible to the client device; and
means for providing the first speech recognition results to the requester in response to the request if the second automatic speech recognition engine is determined not to be accessible to the client device.

### Claim 5

A computer-implemented method performed by a client device, the method comprising:
(A) receiving a request from a requester to apply automatic speech recognition to an audio signal;
(B) providing the audio signal to a first automatic speech recognition engine in a server device;
(C) receiving first speech recognition results from the first automatic speech recognition engine;
(D) determining whether a second automatic speech recognition engine, in the client device, is accessible to the client device;
(E) if the second automatic speech recognition engine is determined not to be accessible to the client device, then providing the first speech recognition results to the requester in response to the request.

### Claim 6

An apparatus comprising:
means for receiving a request from a requester to apply automatic speech recognition to an audio signal;
means for providing the audio signal to a first automatic speech recognition engine in a server device;
means for receiving first speech recognition results from the first automatic speech recognition engine;
means for determining whether a second automatic speech recognition engine, in the client device, is accessible to the client device;
means for providing the first speech recognition results to the requester in response to the request if the second automatic speech recognition engine is determined not to be accessible to the client device.

### Claim 7

A computer-implemented method performed by a client device, the method comprising:
(A) receiving a request from a requester to apply automatic speech recognition to an audio signal;
(B) providing the audio signal to a first automatic speech recognition engine in the client device;
(C) providing the audio signal to a second automatic speech recognition engine in a server device;
(D) receiving first speech recognition results from the first automatic speech recognition engine;
(E) determining whether a confidence measure associated with the first speech recognition results exceeds a predetermined threshold; and
(F) if the confidence measure exceeds the predetermined threshold, then providing the first speech recognition results to the requester in response to the request.

### Claim 8 (depends on 10)

The method of claim 10, further comprising:
(G) before (F), receiving second speech recognition results from the second automatic speech recognition engine; and
wherein (F) comprises providing the first speech recognition results but not the second speech recognition results to the requester.

### Claim 9

An apparatus comprising:
means for receiving a request from a requester to apply automatic speech recognition to an audio signal;
means for providing the audio signal to a first automatic speech recognition engine in the client device;
means for providing the audio signal to a second automatic speech recognition engine in a server device;
means for receiving first speech recognition results from the first automatic speech recognition engine;
means for determining whether a confidence measure associated with the first speech recognition results exceeds a predetermined threshold; and
means for providing the first speech recognition results to the requester in response to the request if the confidence measure exceeds the predetermined threshold.

### Claim 10

A computer-implemented method performed by a client device, the method comprising:
(A) receiving a request from a requester to apply automatic speech recognition to an audio signal;
(B) providing the audio signal to a first automatic speech recognition engine in the client device;
(C) receiving first speech recognition results from the first automatic speech recognition engine;
(D) providing the audio signal to a second automatic speech recognition engine in a server device;
(E) receiving second speech recognition results from the second automatic speech recognition engine;
(F) producing hybrid speech recognition results based on the first speech recognition results and the second speech recognition results; and
(G) providing the hybrid speech recognition results to the requester in response to the request.

### Claim 11 (depends on 10)

The method of claim 10, wherein (F) comprises combining the first speech recognition results and the second speech recognition results using Recognizer Output Voting Error Reduction.

### Claim 12 (depends on 10)

The method of claim 10, wherein the client device is configured to treat one of the first and second automatic speech recognition engines as a preferred speech recognition engine, and:
wherein (C) comprises receiving the first speech recognition results at an arbitration engine in the client device at a first time;
wherein (E) comprises receiving the second speech recognition results at the arbitration engine in the client device at a second time that is later than the first time; and
wherein (F) comprises:
(F)(1) including the first recognition results in the hybrid speech recognition results; and
(F)(2) including the second speech recognition results in the hybrid speech recognition results only if the second automatic speech recognition engine is the preferred speech recognition engine.

### Claim 13 (depends on 10)

The method of claim 10, wherein the client device is configured to treat one of the first and second automatic speech recognition engines as a preferred speech recognition engine, and:
wherein (E) comprises receiving the second speech recognition results at an arbitration engine in the client device at a first time;
wherein (C) comprises receiving the first speech recognition results at the arbitration engine in the client device at a second time that is later than the first time; and
wherein (F) comprises:
(F)(1) including the second recognition results in the hybrid speech recognition results; and
(F)(2) including the first speech recognition results in the hybrid speech recognition results only if the first automatic speech recognition engine is the preferred speech recognition engine.

### Claim 14 (depends on 10)

The method of claim 10:
wherein (C) comprises receiving the first speech recognition results at an arbitration engine in the client device at a first time;
wherein (E) comprises receiving the second speech recognition results at the arbitration engine in the client device at a second time that is later than the first time; and
wherein (F) comprises:
(F)(1) including the first recognition results in the hybrid speech recognition results;
(F)(2) identifying words in the second speech recognition results that do not overlap in time with any words in the first speech recognition results; and
(F)(3) including only non-overlapping words from the second speech recognition results in the hybrid speech recognition results.

### Claim 15 (depends on 10)

The method of claim 10:
wherein (E) comprises receiving the second speech recognition results at an arbitration engine in the client device at a first time;
wherein (C) comprises receiving the first speech recognition results at the arbitration engine in the client device at a second time that is later than the first time; and
wherein (F) comprises:
(F)(1) including the second recognition results in the hybrid speech recognition results;
(F)(2) identifying words in the first speech recognition results that do not overlap in time with any words in the second speech recognition results; and
(F)(3) including only non-overlapping words from the first speech recognition results in the hybrid speech recognition results.

### Claim 16 (depends on 10)

The method of claim 10, wherein (F) comprises:
(F)(1) including the first recognition results in the hybrid speech recognition results; and
(F)(2) replacing the first recognition results with the second recognition results in the hybrid speech recognition results.

### Claim 17 (depends on 10)

The method of claim 10, wherein (F) comprises:
(F)(1) including the second recognition results in the hybrid speech recognition results; and
(F)(2) replacing the second recognition results with the first recognition results in the hybrid speech recognition results.

### Claim 18

An apparatus comprising:
means for receiving a request from a requester to apply automatic speech recognition to an audio signal;
means for providing the audio signal to a first automatic speech recognition engine in the client device;
means for receiving first speech recognition results from the first automatic speech recognition engine;
means for providing the audio signal to a second automatic speech recognition engine in a server device;
means for receiving second speech recognition results from the second automatic speech recognition engine;
means for producing hybrid speech recognition results based on the first speech recognition results and the second speech recognition results; and
means for providing the hybrid speech recognition results to the requester in response to the request.
