# US 2019/0378495 A1 · Artificial intelligence apparatus for recognizing speech of user using personalized language model and method for the same

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0378495 A1
- Document ID: US20190378495A1
- Status: Published application
- Kind: A1
- Application number: 16/546,924
- Filed: 2019-08-21
- Published: 2019-12-12
- Earliest claimed: 2019-07-23
- Assignees: LG Electronics Inc.
- Applicants: LG Electronics Inc.
- Inventors: Boseop KIM; Jaehong KIM
- Web page: https://patentgrep.com/patent/US20190378495A1

## CPC

- G10L 15/07: to the speaker
- G10L 15/183: using context dependencies, e.g. language models
- G10L 15/02: Feature extraction for speech recognition; Selection of recognition unit
- G10L 15/063: {Training}
- G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning}
- G10L 15/187: Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L 15/193: Formal grammars, e.g. finite state automata, context free grammars or word networks
- G10L 2015/025: {Phonemes, fenemes or fenones being the recognition units}
- G10L 2015/0635: {updating or merging of old and new templates; Mean values; Weighting}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

An artificial intelligence apparatus for recognizing speech of a user includes a microphone, and a processor configured to receive, via the microphone, a sound signal corresponding to the speech of the user, acquire personalize identification information corresponding to the speech, recognize the speech from the sound signal using a global language model, calculate a reliability for the recognition, and if the calculated reliability exceeds a predetermined first reference value, update a personalized language model corresponding to the personalize identification information using the recognition result.

## Claims

### Claim 1

An artificial intelligence apparatus for recognizing speech of a user, comprising:
a microphone; and
a processor configured to:
receive, via the microphone, a sound signal corresponding to the speech of the user;
acquire personalize identification information corresponding to the speech;
recognize the speech from the sound signal using a global language model;
calculate a reliability for the recognition; and
if the calculated reliability exceeds a predetermined first reference value, update a personalized language model corresponding to the personalize identification information using the recognition result.

### Claim 2 (depends on 1)

The artificial intelligence apparatus according to claim 1, wherein the personalized language model is a language model configured to generate a word vector from a personalize vector representing the personalize identification information and phonemes recognized in an acoustic model, and
the processor is configured to update the personalized language model corresponding to the personalize identification information by increasing a weight of a word included in the speech recognition result.

### Claim 3 (depends on 2)

The artificial intelligence apparatus according to claim 2, wherein the processor is configured to calculate a first language model (LM) score corresponding to the speech recognition result as the reliability of the recognition.

### Claim 4 (depends on 3)

The artificial intelligence apparatus according to claim 3, wherein if the calculated reliability does not exceed the first reference value, the processor is configured to extract a misrecognized word lowering the first LM score from the speech recognition result and correct the misrecognized word using the personalized language model corresponding to the personalize identification information.

### Claim 5 (depends on 4)

The artificial intelligence apparatus according to claim 4, wherein the processor is configured to:
calculate a similarity between the extracted misrecognized word and the personalize vector;
if the calculated similarity exceeds a predetermined threshold, determine a word most similar to the misrecognized word in a lexicon; and
correct the misrecognized word by replacing the misrecognized word with the most similar word.

### Claim 6 (depends on 4)

The artificial intelligence apparatus according to claim 4, wherein the processor is configured to:
calculate a second LM score for each word in the speech recognition result; and
extract words for which the second LM score does not exceed a predetermined second reference value as the misrecognized word.

### Claim 7 (depends on 4)

The artificial intelligence apparatus according to claim 4, wherein the processor is configured to:
calculate a third LM score with respect to the corrected recognition result; and
if the calculated third LM score exceeds a predetermined third reference value, determine that the speech recognition is successful, and update the personalized language model corresponding to the personalize identification information using the corrected recognition result.

### Claim 8 (depends on 1)

The artificial intelligence apparatus according to claim 1, wherein the processor is configured to:
if the calculated reliability exceeds the first reference value, determine an intention corresponding to the speech recognition result; and
perform an operation corresponding to the determined intention.

### Claim 9 (depends on 8)

The artificial intelligence apparatus according to claim 8, wherein the processor is configured to:
project the recognition result in a vector space using an intention classifier; and
determine the intention of the user by comparing a position of the projected recognition result with positions of a plurality of intention groups included in the vector space.

### Claim 10 (depends on 9)

The artificial intelligence apparatus according to claim 9, wherein the processor is configured to determine the intention of the user as an intention corresponding to a nearest intention group nearest from the position of the projected recognition result among the plurality of intention groups.

### Claim 11 (depends on 1)

The artificial intelligence apparatus according to claim 1, wherein the personalize identification information includes at least one of user identification information for distinguishing each user or device identification information for distinguishing each device,
the user identification information is information indicating a user identified according to voice analysis of the speech, and
the device identification information is information indicating a device that has received the speech.

### Claim 12 (depends on 1)

The artificial intelligence apparatus according to claim 1, wherein the global language model and the personalized language model are a model learned using a machine learning algorithm or a deep learning algorithm, and configured as an artificial neural network.

### Claim 13

A method for recognizing speech of a user, comprising:
receiving a sound signal corresponding to speech of the user;
acquiring personalize identification information corresponding to the speech;
recognizing the speech from the sound signal using a global language model;
calculating reliability of the recognition; and
if the calculated reliability exceeds a predetermined first reference value, updating a personalized language model corresponding to the personalize identification information using the speech recognition result.

### Claim 14

A recording medium having recorded thereon a program for performing a method for recognizing speech of a user, the method comprising:
receiving a sound signal corresponding to speech of the user;
acquiring personalize identification information corresponding to the speech;
recognizing the speech from the sound signal using a global language model;
calculating reliability of the recognition; and
if the calculated reliability exceeds a predetermined first reference value, updating a personalized language model corresponding to the personalize identification information using the speech recognition result.