# US 2021/0174789 A1 · Automatic speech recognition device and method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0174789 A1
- Document ID: US20210174789A1
- Status: Published application
- Kind: A1
- Application number: 16/763,901
- Filed: 2018-11-06
- Published: 2021-06-10
- Earliest claimed: 2017-11-14
- Assignees: Llsollu Co., Ltd.
- Applicants: Llsollu Co., Ltd.
- Inventors: Myeongjin HWANG; Changjin JI
- Web page: https://patentgrep.com/patent/US20210174789A1

## CPC

- G10L 15/187: Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
- G10L 15/063: {Training}
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L 15/06: Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice (G10L15/14 takes precedence)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

An automatic speech recognition device, according to the present invention, comprises: a memory for storing a program for converting speech data received via an interface module into transcription data, and outputting same; and a processor for executing the program stored in the memory, wherein, by executing the program, the processor converts the received speech data into pronunciation code data on the basis of a pre-trained first model, and converts the converted pronunciation code data into transcription data on the basis of a pre-trained second model.

## Claims

### Claim 1

An automatic speech recognition device comprising:
a memory configured to store a program for converting speech data received through an interface module into transcription data and outputting the transcription data; and
a processor configured to execute the program stored in the memory,
wherein, by executing the program, the processor converts the received speech data into pronunciation code data based on a pre-trained first model, and converts the pronunciation code data into transcription data based on a pre-trained second model.

### Claim 2 (depends on 1)

The automatic speech recognition device of claim 1, wherein the pre-trained first model includes a speech-pronunciation code conversion model and the speech-pronunciation code conversion model is trained based on parallel data composed of the speech data and the pronunciation code data.

### Claim 3 (depends on 2)

The automatic speech recognition device of claim 2, wherein the converted pronunciation code data includes a feature value sequence of a phoneme or sound having a length of 1 or more that is expressible in a one-dimensional structure.

### Claim 4 (depends on 2)

The automatic speech recognition device of claim 2, wherein the converted pronunciation code data includes a language-independent value.

### Claim 5 (depends on 1)

The automatic speech recognition device of claim 1, wherein the pre-trained second model includes a pronunciation code-transcription conversion model, and the pronunciation code-transcription conversion model is trained based on parallel data composed of the pronunciation code data and the transcription data.

### Claim 6 (depends on 1)

The automatic speech recognition device of claim 1, wherein the pre-trained second model includes a pronunciation code-transcription conversion model, and the second model converts a sequence type pronunciation code into a sequence type transcription at a time.

### Claim 7 (depends on 1)

The automatic speech recognition device of claim 1, wherein the pre-trained first model includes a speech-pronunciation code conversion model and the speech-pronunciation code conversion model is generated by performing unsupervised learning based on previously prepared speech data.

### Claim 8 (depends on 7)

The automatic speech recognition device of claim 7, wherein the previously prepared speech data is constructed as parallel data together with the transcription data.

### Claim 9 (depends on 8)

The automatic speech recognition device of claim 8, wherein the pre-trained second model includes a pronunciation code-transcription conversion model, the processor is configured to convert the speech data into the pronunciation code data to correspond to the speech data included in the parallel data based on a pre-trained speech-pronunciation code conversion model, and the pre-trained speech-pronunciation code conversion model is trained based on parallel data including the pronunciation code data converted corresponding to the speech data by the processor and the transcription data.

### Claim 10 (depends on 2, 7)

The automatic speech recognition device of claim 2 or 7, wherein the processor generates a candidate sequence of characters from the converted pronunciation code data by using pre-prepared syllable-pronunciation dictionary data, and converts the candidate string of characters generated through the second model, which is a language model trained based on corpus data, into the transcription data.

### Claim 11

An automatic speech recognition method comprising:
receiving speech data;
converting the received speech data into a pronunciation code sequence based on a pre-trained first model; and
converting the converted pronunciation code sequence into transcription data based on a pre-trained second model.
