# US 2022/0068269 A1 · Adaptive batching to reduce recognition latency

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0068269 A1
- Document ID: US20220068269A1
- Status: Published application
- Kind: A1
- Application number: 17/500,585
- Filed: 2021-10-13
- Published: 2022-03-03
- Earliest claimed: 2020-01-13
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Hosam A. KHALIL; Emilian Y. STOIMENOV; Yifan GONG; Chaojun LIU; Christopher H. BASOGLU; Amit K. AGARWAL; Naveen PARIHAR; Sayan PATHAK
- Web page: https://patentgrep.com/patent/US20220068269A1

## CPC

- G10L 15/197: Probabilistic grammars, e.g. word n-grams
- G10L 15/32: Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems
- G10L 15/02: Feature extraction for speech recognition; Selection of recognition unit
- G10L 15/05: Word boundary detection
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L 15/30: Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
- G10L 15/16: using artificial neural networks
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

Embodiments may include collection of a first batch of acoustic feature frames of an audio signal, the number of acoustic feature frames of the first batch equal to a first batch size, input of the first batch to a speech recognition network, collection, in response to detection of a word hypothesis output by the speech recognition network, of a second batch of acoustic feature frames of the audio signal, the number of acoustic feature frames of the second batch equal to a second batch size greater than the first batch size, and input of the second batch to the speech recognition network.

## Claims

### Claim 1

A system comprising:
a processing unit; and
a storage device including program code that when executed by the processing unit causes the system to:
collect a first batch comprising a first number of raw acoustic feature frames of the audio signal, the first number equal to a first batch size;
input the first batch to a speech recognition network;
in response to a word hypothesis output by the speech recognition network, collect a second batch comprising a second number of acoustic feature frames of the audio signal, the second number equal to a second batch size; and
input the second batch to the speech recognition network.

### Claim 2 (depends on 1)

A system according to claim 1, wherein collection of the second batch is in response to detection of a first non-silent word hypothesis output by the speech recognition network.

### Claim 3 (depends on 1)

A system according to claim 1, the program code that when executed by the processing unit causes the system to:
collect a plurality of batches, each of the plurality of batches comprising the first number of acoustic feature frames of the audio signal;
input the plurality of batches to the speech recognition network until the word hypothesis is output by the speech recognition network.

### Claim 4 (depends on 1)

A system according to claim 1, the program code that when executed by the processing unit causes the system to:
detect an end-of-speech frame; and
reduce a size of a next collected batch of acoustic feature frames to include the end-of-speech frame as a last frame of the next collected batch.

### Claim 5 (depends on 1)

A system according to claim 1, wherein the first batch size comprises a first number of look-ahead frames and the first batch comprises first look-ahead frames, the number of first look-ahead frames of the first batch equal to the first number of look-ahead frames, the program code that when executed by the processing unit causes the system to:
collect a third batch comprising the first number of acoustic feature frames of the audio signal, a time period associated with a plurality of frames of the third batch overlapping a time period associated with the first look-ahead frames of the first batch, and
wherein the second batch comprises second look-ahead frames, the number of second look-ahead frames of the second batch equal to a second number of look-ahead frames.

### Claim 6 (depends on 5)

A system according to claim 5, wherein the speech recognition network comprises a latency-controlled bi-directional long short-term memory model.

### Claim 7 (depends on 1)

A system according to claim 1, wherein the first batch size comprises a first number of look-ahead frames and the first batch comprises first look-ahead frames, the number of first look-ahead frames of the first batch equal to the first number of look-ahead frames, the program code that when executed by the processing unit causes the system to:
collect a third batch comprising the first number of acoustic feature frames of the audio signal, a time period associated with a plurality of frames of the third batch overlapping a time period associated with the first look-ahead frames of the first batch,
wherein the second batch comprises second look-ahead frames, the number of second look-ahead frames of the second batch equal to the first number of look-ahead frames.

### Claim 8 (depends on 7)

A system according to claim 7, wherein the speech recognition network comprises a contextual layer-trajectory long short-term memory model.

### Claim 9

A computer-implemented method comprising:
collecting a first batch of acoustic feature frames of an audio signal, the number of acoustic feature frames of the first batch equal to a first batch size;
inputting the first batch to a speech recognition network;
in response to detection of a word hypothesis output by the speech recognition network, collecting a second batch of acoustic feature frames of the audio signal, the number of acoustic feature frames of the second batch equal to a second batch size; and
inputting the second batch to the speech recognition network.

### Claim 10 (depends on 9)

A method according to claim 9, wherein detection of the word hypothesis comprises detection of a first non-silent word hypothesis output by the speech recognition network.

### Claim 11 (depends on 9)

A method according to claim 9, further comprising:
detecting an end-of-speech frame;
in response to detecting the end-of-speech frame, collecting a third batch of acoustic feature frames of the audio signal, the third batch to include the end-of-speech frame as a last frame of the third batch; and
inputting the third batch to the speech recognition network.

### Claim 12 (depends on 9)

A method according to claim 9, wherein the first batch size comprises a first number of look-ahead frames and the first batch comprises first look-ahead frames, the number of first look-ahead frames of the first batch equal to the first number of look-ahead frames, the method further comprising:
collecting a third batch comprising the first number of acoustic feature frames of the audio signal, a time period associated with a plurality of frames of the third batch overlapping a time period associated with the first look-ahead frames of the first batch,
wherein the second batch comprises second look-ahead frames, the number of second look-ahead frames of the second batch equal to the second number of look-ahead frames.

### Claim 13 (depends on 12)

A method according to claim 12, wherein the speech recognition network comprises a latency-controlled bi-directional long short-term memory model.

### Claim 14 (depends on 9)

A method according to claim 9, wherein the first batch size comprises a first number of look-ahead frames and the first batch comprises first look-ahead frames, the number of first look-ahead frames of the first batch equal to the first number of look-ahead frames, further comprising:
collecting a third batch comprising the first number of acoustic feature frames of the audio signal, a time period associated with a plurality of frames of the third batch overlapping a time period associated with the first look-ahead frames of the first batch,
wherein the second batch comprises second look-ahead frames, the number of second look-ahead frames of the second batch equal to the first number of look-ahead frames.

### Claim 15 (depends on 14)

A method according to claim 14, wherein the speech recognition network comprises a contextual layer-trajectory long short-term memory model.

### Claim 16

A non-transient computer-readable medium storing program code to be executed by a processing unit to provide:
a speech recognition network;
a batcher to:
collect a first batch of acoustic feature frames of an audio signal, the number of acoustic feature frames of the first batch equal to a first batch size;
input the first batch to the speech recognition network;
in response to detection of a word hypothesis output by the speech recognition network, collect a second batch of acoustic feature frames of the audio signal, the number of acoustic feature frames of the second batch equal to a second batch size; and
input the second batch to the speech recognition network.

### Claim 17 (depends on 16)

A non-transient computer-readable medium according to claim 16, storing program code to be executed by a processing unit to provide:
an end-of-speech detector to:
receive acoustic feature frames of the audio signal independent of reception of acoustic feature frames by the batcher; and
detect an end-of-speech frame based on the received acoustic feature frames;
wherein response to detection of the end-of-speech frame, the batcher is further to collect a third batch of acoustic feature frames of the audio signal, the third batch to include the end-of-speech frame as a last frame of the third batch, and input the third batch to the speech recognition network.

### Claim 18 (depends on 16)

A non-transient computer-readable medium according to claim 16, wherein the first batch size comprises a first number of look-ahead frames and the first batch comprises first look-ahead frames, the number of first look-ahead frames of the first batch equal to the first number of look-ahead frames, wherein the batcher is further to:
collect a third batch comprising the first number of acoustic feature frames of the audio signal, a time period associated with a plurality of frames of the third batch overlapping a time period associated with the first look-ahead frames of the first batch,
wherein the second batch comprises second look-ahead frames, the number of second look-ahead frames of the second batch equal to the second number of look-ahead frames, and
wherein the speech recognition network comprises a latency-controlled bi-directional long short-term memory model.

### Claim 19 (depends on 16)

A non-transient computer-readable medium according to claim 16, wherein the first batch size comprises a first number of look-ahead frames and the first batch comprises first look-ahead frames, the number of first look-ahead frames of the first batch equal to the first number of look-ahead frames, wherein the batcher is further to:
collect a third batch comprising the first number of acoustic feature frames of the audio signal, a time period associated with a plurality of frames of the third batch overlapping a time period associated with the first look-ahead frames of the first batch,
wherein the second batch comprises second look-ahead frames, the number of second look-ahead frames of the second batch equal to the first number of look-ahead frames, and
wherein the speech recognition network comprises a contextual layer-trajectory long short-term memory model.

### Claim 20 (depends on 16)

A non-transient computer-readable medium according to claim 16, wherein detection of the word hypothesis comprises detection of a first non-silent word hypothesis output by the speech recognition network.
