# US 2021/0174782 A1 · Artificial intelligence device and method for synthesizing speech by controlling speech style

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0174782 A1
- Document ID: US20210174782A1
- Status: Published application
- Kind: A1
- Application number: 16/803,941
- Filed: 2020-02-27
- Published: 2021-06-10
- Earliest claimed: 2019-12-09
- Assignees: LG Electronics Inc.
- Applicants: LG Electronics Inc.
- Inventors: Minook KIM; Yongchul PARK; Sungmin HAN; Siyoung YANG; Sangki KIM; Juyeong JANG
- Web page: https://patentgrep.com/patent/US20210174782A1

## CPC

- G10L 13/033: Voice editing, e.g. manipulating the voice of the synthesiser
- G10L 13/10: Prosody rules derived from text; Stress or intonation
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/09: Supervised learning
- G06N 5/04: Inference or reasoning models
- G06N 20/00: Machine learning
- G10L 13/027: Concept to speech synthesisers; Generation of natural phrases from machine-based concepts (generation of parameters for speech synthesis out of text G10L13/08)
- G10L 13/047: Architecture of speech synthesisers
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/045: Combinations of networks
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

An artificial intelligence device includes a memory and a processor. The memory is configured to store audio data having a predetermined speech style. The processor is configured to generate a condition vector relating to a condition for determining the speech style of the audio data, reduce a dimension of the condition vector to a predetermined reduction dimension, acquire a sparse code vector based on a dictionary vector acquired through sparse dictionary coding with respect to the condition vector having the predetermined reduction dimension, and change a vector element value included in the sparse code vector.

## Claims

### Claim 1

A method for controlling a speech style, the method comprising:
acquiring audio data having a predetermined speech style;
generating a condition vector relating to a condition for determining the speech style of the audio data;
reducing a dimension of the condition vector to a predetermined reduction dimension;
acquiring a sparse code vector based on a dictionary vector acquired through sparse dictionary coding with respect to the condition vector having the predetermined reduction dimension; and
changing a vector element value included in the sparse code vector.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the reducing to the predetermined reduction dimension comprises reducing the condition vector to the predetermined reduction dimension by applying a Principal Component Analysis (PCA) algorithm to the condition vector.

### Claim 3 (depends on 1)

The method according to claim 1, further comprising:
acquiring a plurality of pieces of audio training data for the sparse dictionary coding;
acquiring condition training vectors relating to the condition for determining the speech style with respect to the plurality of pieces of audio training data;
reducing the dimension of each of the condition training vectors to the predetermined reduction dimension; and
acquiring a dictionary vector and a sparse representation coefficient vector, which are capable of acquiring the condition training vector, through sparse coding.

### Claim 4 (depends on 1)

The method according to claim 1, wherein the changing of the vector element value comprises changing a vector element value for a valid vector element included in the sparse code vector.

### Claim 5 (depends on 1)

The method according to claim 1, wherein, when the sparse code vector is plural, the changing of the vector element value comprises changing a vector element value based on a valid vector element included in each of the plurality of sparse code vectors.

### Claim 6 (depends on 1)

The method according to claim 1, further comprising:
acquiring the condition vector having the predetermined reduction dimension from the sparse code vector having the changed vector element value based on the dictionary vector; and
acquiring the condition vector in which the condition for determining the speech style is changed by extending the dimension of the condition vector having the predetermined dimension.

### Claim 7 (depends on 6)

The method according to claim 6, further comprising:
acquiring a prosody vector representing each of at least one speech style; and
generating a prosody embedding vector having a changed speech style using the prosody vector and the condition vector having the changed condition for determining the speech style.

### Claim 8 (depends on 7)

The method according to claim 7, further comprising:
acquiring text data; and
generating a synthesized speech based on the text data and the prosody embedding vector.

### Claim 9

An artificial intelligence device comprising:
a memory configured to store audio data having a predetermined speech style; and
a processor configured to:
generate a condition vector relating to a condition for determining the speech style of the audio data;
reduce a dimension of the condition vector to a predetermined reduction dimension;
acquire a sparse code vector based on a dictionary vector acquired through sparse dictionary coding with respect to the condition vector having the predetermined reduction dimension; and
change a vector element value included in the sparse code vector.

### Claim 10 (depends on 9)

The artificial intelligence device according to claim 9, wherein the processor is configured to reduce the condition vector to the predetermined reduction dimension by applying a Principal Component Analysis (PCA) algorithm to the condition vector.

### Claim 11 (depends on 9)

The artificial intelligence device according to claim 9, wherein the processor is configured to:
acquire a plurality of pieces of audio training data for the sparse dictionary coding;
acquire condition training vectors for determining the speech style with respect to the plurality of pieces of audio training data;
reduce the dimension of each of the condition training vectors to the predetermined reduction dimension; and
acquire a dictionary vector and a sparse representation coefficient vector, which are capable of acquiring the condition training vector, through sparse coding.

### Claim 12 (depends on 9)

The artificial intelligence device according to claim 9, wherein the processor is configured to change a vector element value for a valid vector element included in the sparse code vector.

### Claim 13 (depends on 9)

The artificial intelligence device according to claim 9, wherein, when the sparse code vector is plural, the processor is configured to change a vector element value based on a valid vector element included in each of the plurality of sparse code vectors.

### Claim 14 (depends on 9)

The artificial intelligence device according to claim 9, wherein the processor is configured to:
acquire the condition vector having the predetermined reduction dimension from the sparse code vector having the changed vector element value based on the dictionary vector; and
acquire the condition vector in which the condition for determining the speech style is changed by extending the dimension of the condition vector having the predetermined dimension.

### Claim 15 (depends on 14)

The artificial intelligence device according to claim 14, wherein the processor is configured to:
acquire a prosody vector representing each of at least one speech style; and
generate a prosody embedding vector having a changed speech style using the prosody vector and the condition vector having the changed condition for determining the speech style.

### Claim 16 (depends on 15)

The artificial intelligence device according to claim 15, wherein the processor is configured to:
acquire text data; and
generate a synthesized speech based on the text data and the prosody embedding vector.
