# US 12,217,738 B2 · Method for generating training data and method for post-processing of speech recognition using the same

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,217,738 B2
- Document ID: US12217738B2
- Status: Granted
- Kind: B2
- Application number: 17/739,383
- Filed: 2022-05-09
- Published: 2022-11-17
- Granted: 2025-02-04
- Earliest claimed: 2021-05-11
- Assignees: Korea University Research and Business Foundation
- Applicants: Korea University Research and Business Foundation
- Inventors: Heuiseok Lim; Chanjun Park
- Web page: https://patentgrep.com/patent/US12217738B2

## CPC

- G06F 40/253: Grammatical analysis; Style critique
- G10L 15/063: {Training}
- G06F 40/166: Editing, e.g. inserting or deleting
- G06F 40/216: using statistical methods
- G06F 40/232: Orthographic correction, e.g. spell checking or vowelisation
- G10L 13/00: Speech synthesis; Text to speech systems
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

Disclosed is a training data construction method and a speech recognition method using the same. The training data construction method is performed by a computing apparatus including at least one processor and includes converting first text data including a plurality of sentences to first speech data; acquiring second speech data by adding noise to the first speech data; and converting the second speech data to second text data. The second text data includes a sentence corresponding to each of the plurality of sentences included in the first text data.

## Claims

### Claim 1

A training data construction method performed by a computing apparatus comprising at least one processor, the training data construction method comprising:
converting first text data, which is an initial text of mono corpus including a plurality of sentences, to first speech data using a predetermined text-to-speech scheme;
generating second speech data based on the first speech data, the generating comprising:
adding noise to the first speech data;
performing a frequency transformation on the first speech data;
removing a portion of a predetermined frequency band of the first speech data; and
performing a retransformation to a time domain to generate the second speech data; and
converting the second speech data to second text data, which is a final text,
wherein the second text data includes a plurality of sentences corresponding to each of the plurality of sentences included in the first text data, thereby constituting a parallel corpus with the first text data.

### Claim 2 (depends on 1)

The training data construction method of claim 1, wherein the noise includes natural noise including rain or thunder, or pollution noise including ambulance sound or patrol car sound.

### Claim 3 (depends on 1)

The training data construction method of claim 1, wherein the performing of the frequency transformation on the first speech data comprises using a Fourier transform (FT), a short-time Fourier transform (STFT), or a wavelet Transform.

### Claim 4

A speech recognition method performed by a computing apparatus comprising at least one processor, the speech recognition method comprising:
generating training data;
generating a speech recognition postprocessing model by training a sequence-to-sequence (seq2seq) model using the training data;
generating a text sentence corresponding to input speech data using a predetermined speech recognition model; and
correcting the text sentence using the speech recognition postprocessing model and outputting the corrected text sentence,
wherein the generating of the training data comprises:
converting first text data, which is an initial text of mono corpus including a plurality of sentences, to first speech data using a predetermined text-to-speech scheme;
generating second speech data based on the first speech data, the generating comprising:
adding noise to the first speech data;
performing a frequency transformation on the first speech data;
removing a portion of a predetermined frequency band of the first speech data; and
performing a retransformation to a time domain to generate the second speech data; and
converting the second speech data to second text data, which is a final text,
wherein the second text data includes a plurality of sentences corresponding to each of the plurality of sentences included in the first text data, thereby constituting a parallel corpus with the first text data.

### Claim 5 (depends on 4)

The speech recognition method of claim 4, wherein the noise comprises natural noise including rain or thunder, or pollution noise including ambulance sound or patrol car sound.

### Claim 6 (depends on 5)

The speech recognition method of claim 5, wherein the training data includes the first text data and the second text data.