# US 2026/0073164 A1 · Multi-task automatic speech recognition system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0073164 A1
- Document ID: US20260073164A1
- Status: Published application
- Kind: A1
- Application number: 19/394,314
- Filed: 2025-11-19
- Published: 2026-03-12
- Earliest claimed: 2023-04-18
- Assignees: OpenAI OpCo, LLC
- Applicants: OpenAI Opco, LLC
- Inventors: Alec RADFORD; Jong Wook KIM; Tao XU; Greg BROCKMAN; Christine MCLEAVEY-PAYNE; Ilya SUTSKEVER
- Web page: https://patentgrep.com/patent/US20260073164A1

## CPC

- G06F 40/58: Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Disclosed herein are methods, systems, and computer-readable media for generating an output transcript from an input audio segment using a multi-task transformer model. In some embodiments, the transformer model can be trained to transcribe or translate audio data in multiple languages using labeled audio data. The labeled audio data can include first audio segments associated with first same-language transcripts of the first audio segments and second audio segments associated with second different-language transcripts of the second audio segments. In some embodiments, a vocabulary of the model can include special purpose and time stamp tokens. The special purpose tokens can specify tasks for the model to perform.

## Claims

### Claim 1

A system comprising:
at least one memory storing instructions;
at least one processor configured to execute the instructions to perform operations for multi-language, multi-task speech recognition, the operations comprising:
obtaining a transformer model including an encoder and a decoder, the transformer model trained to transcribe or translate audio data in multiple languages using labeled audio data, the labeled audio data including first audio segments associated with first same-language transcripts of the first audio segments and second audio segments associated with second different-language transcripts of the second audio segments; and
generating an output transcript from an input audio segment using the transformer model, generation including:
configuring a decoder input with a language token corresponding to a first language; and
configuring the decoder input with a task token.

### Claim 2 (depends on 1)

The system of claim 1, wherein the task token is a transcribe token; or a translate token associated with a second language differing from the first language.

### Claim 3 (depends on 1)

The system of claim 1, wherein:
the task token is a translate token; and
generating the output transcript further includes:
autoregressively configuring the decoder input with a textual token predicted using the translate token, the textual token associated with a second language differing from the first language.

### Claim 4 (depends on 1)

The system of claim 1, wherein the decoder is configured with a vocabulary including timestamp tokens.

### Claim 5 (depends on 1)

The system of claim 1, wherein generating the output transcript further includes:
autoregressively configuring the decoder input with a first timestamp token predicted by the decoder based on an absence of a notimestamp token in the decoder input.

### Claim 6 (depends on 1)

The system of claim 1, wherein generating the output transcript further includes:
autoregressively configuring the decoder input with a predicted no speech token followed by an end of transcript token.

### Claim 7 (depends on 1)

The system of claim 1, wherein the transformer model is configured to perform inverse text normalization.

### Claim 8 (depends on 1)

The system of claim 1, wherein generating the output transcript further includes:
applying a first subsegment of the input audio segment to generate one or more predicted timestamp tokens for the first subsegment; and
generating a second subsegment of the input audio segment using the one or more predicted timestamp tokens for the first subsegment.

### Claim 9 (depends on 1)

The system of claim 1, wherein generating the output transcript further includes performing a beam search using an output softmax temperature dependent on at least one of:
log probabilities of previously generated tokens of the output transcript; or
a gzip compression rate of the previously generated tokens of the output transcript.

### Claim 10 (depends on 1)

The system of claim 1, wherein generating the output transcript further includes prepending a transcript generated for a preceding input audio segment to the decoder input.

### Claim 11

A method for multi-language, multi-task speech recognition, comprising:
obtaining a transformer model including an encoder and a decoder, the transformer model trained to transcribe or translate audio data in multiple languages using labeled audio data, the labeled audio data including first audio segments associated with first same-language transcripts of the first audio segments and second audio segments associated with second different-language transcripts of the second audio segments; and
generating an output transcript from an input audio segment using the transformer model, generation including:
configuring a decoder input with a language token corresponding to a first language; and
configuring the decoder input with a task token.

### Claim 12 (depends on 11)

The method of claim 11, wherein the task token is a transcribe token or a translate token associated with a second language differing from the first language.

### Claim 13 (depends on 11)

The method of claim 11, wherein:
the task token is a translate token; and
generating the output transcript further includes:
autoregressively configuring the decoder input with a textual token predicted using the translate token, the textual token associated with a second language differing from the first language.

### Claim 14 (depends on 11)

The method of claim 11, wherein the decoder is configured with a vocabulary including timestamp tokens.

### Claim 15 (depends on 11)

The method of claim 11, wherein generation further includes autoregressively configuring the decoder input with a first timestamp token predicted by the decoder based on an absence of a notimestamp token in the decoder input.

### Claim 16 (depends on 11)

The method of claim 11, wherein generating the output transcript further includes autoregressively configuring the decoder input with a predicted no speech token followed by an end of transcript token.

### Claim 17 (depends on 11)

The method of claim 11, wherein the transformer model is configured to perform inverse text normalization.

### Claim 18 (depends on 11)

The method of claim 11, wherein generating the output transcript further includes:
applying a first subsegment of the input audio segment to generate one or more predicted timestamp tokens for the first subsegment; and
generating a second subsegment of the input audio segment using the one or more predicted timestamp tokens for the first subsegment.

### Claim 19 (depends on 11)

The method of claim 11, wherein generating the output transcript further includes:
performing a beam search using an output softmax temperature dependent on at least one of:
log probabilities of previously generated tokens of the output transcript; or
a gzip compression rate of the previously generated tokens of the output transcript; or
prepending a transcript generated for a preceding input audio segment to the decoder input.

### Claim 20

A non-transitory computer-readable medium including instructions that are executable by one or more processors to perform operations comprising:
obtaining a transformer model including an encoder and a decoder, the transformer model trained to transcribe or translate audio data in multiple languages using labeled audio data, the labeled audio data including first audio segments associated with first same-language transcripts of the first audio segments and second audio segments associated with second different-language transcripts of the second audio segments; and
generating an output transcript from an input audio segment using the transformer model, generation including:
autoregressively configuring a decoder input with a predicted language token corresponding to a first language; and
configuring the decoder input with a task token.
