# US 10,311,888 B2 · Voice quality conversion device, voice quality conversion method and program

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,311,888 B2
- Document ID: US10311888B2
- Status: Granted
- Kind: B2
- Application number: 16/079,383
- Filed: 2017-02-22
- Published: 2019-02-14
- Granted: 2019-06-04
- Earliest claimed: 2016-02-23
- Assignees: The University of Electro-Communications
- Applicants: The University of Electro-Communications
- Inventors: Toru Nakashika; Yasuhiro Minami
- Web page: https://patentgrep.com/patent/US10311888B2

## CPC

- G10L 21/013: Adapting to target pitch
- G10L 21/003: Changing voice quality, e.g. pitch or formants
- G10L 21/007: characterised by the process used
- G10L 25/21: the extracted parameters being power information
- G10L 2021/0135: {Voice conversion or morphing}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

A voice conversion device includes: a parameter learning unit in which a probabilistic model that uses speech information, speaker information, and phonological information as variables to thereby express relationships among binding energies between any two of the speech information, the speaker information and the phonological information by parameters is prepared, wherein the speech information is obtained based on a speech, the speaker information corresponds to the speech information, and the phonological information expresses the phoneme of the speech, and in which the parameters are determined by performing learning by sequentially inputting the speech information and the speaker information into the probabilistic model; and a voice conversion processing unit that performs voice conversion processing of the speech information obtained on the basis of the speech of an input speaker, based both on the parameters determined by the parameter learning unit and on the speaker information of a target speaker.

## Claims

### Claim 1

A voice conversion device adapted to perform voice conversion to convert the voice of an input speaker into the voice of a target speaker, comprising:
a central processing unit (CPU);
a parameter learning unit, executed by the CPU, in which a probabilistic model that uses speech information, speaker information, and phonological information as variables to thereby express relationships among binding energies between any two of the speech information, the speaker information and the phonological information by parameters is prepared, wherein the speech information is obtained based on a speech, the speaker information corresponds to the speech information, and the phonological information expresses the phoneme of the speech, and in which the parameters are determined by performing learning by sequentially inputting the speech information and the speaker information corresponding to the speech information into the probabilistic model; and
a voice conversion processing unit, executed by the CPU, that performs voice conversion processing of the speech information obtained on the basis of the speech of the input speaker, based both on the parameters determined by the parameter learning unit and on the speaker information of the target speaker.

### Claim 2 (depends on 1)

The voice conversion device according to claim 1,
wherein the parameters are composed of seven parameters which are M, V, U, A, b, c and σ, wherein M expresses the degree of the relationship between the speech information and the phonological information, V expresses the degree of the relationship between the phonological information and the speaker information, U expresses the degree of the relationship between the speaker information and the speech information, A represents a set of projection matrix determined by the speaker information, b represents a bias of the speech information, c represents a bias of the speech information, and σ represents the deviation of the speech information, and
wherein the seven parameters are related to each other by the following Formulas (A) to (D) where v represents the speech information, h represents the phonological information, and s represents the speaker information.
E
⁡
(
v
,
h
,
s
)
=
1
2
⁢
v
⊤
⁢
v
_
-
b
⊤
⁢
v
_
-
c
⊤
⁢
h
-
h
⊤
⁢
Vs
-
s
⊤
⁢
U
⁢
v
_
-
v
_
⊤
⁢
A
s
⁢
Mh
,
(
A
)
p
⁡
(
v
|
h
,
s
)
=
𝒩
⁡
(
v
|
b
+
U
⊤
⁢
s
+
A
s
⁢
Mh
,
σ
2
)
(
B
)
p
⁡
(
h
|
s
,
v
)
=
ℬ
⁡
(
h
|
f
⁡
(
c
+
Vs
+
M
⊤
⁢
A
s
⊤
⁢
v
_
)
)
(
C
)
p
⁡
(
s
|
v
,
h
)
=
ℬ
⁡
(
s
|
f
⁡
(
U
⁢
v
_
+
V
⊤
⁢
h
+
[
v
_
⊤
⁢
A
k
]
⁢
Mh
)
)
.
(
D
)

### Claim 3

A voice conversion method for performing voice conversion to convert the voice of an input speaker to the voice of a target speaker, comprising:
a parameter learning step in which a probabilistic model that uses speech information, speaker information, and phonological information as variables to thereby express relationships among binding energies between any two of the speech information, the speaker information and the phonological information by parameters is prepared, wherein the speech information is obtained based on a speech, the speaker information corresponds to the speech information, and the phonological information expresses the phoneme of the speech, and in which the parameters are determined by performing learning by sequentially inputting the speech information and the speaker information corresponding to the speech information into the probabilistic model; and
a voice conversion processing step of performing voice conversion processing of the speech information obtained on the basis of the speech of the input speaker, based both on the parameters determined in the parameter learning step and on the speaker information of the target speaker.

### Claim 4

A non-transitory computer readable medium embodying a program that, when executed by a central processing unit (CPU), causes a computer to execute a method, the method comprising:
a parameter learning step in which a probabilistic model that uses speech information, speaker information, and phonological information as variables to thereby express relationships among binding energies between any two of the speech information, the speaker information and the phonological information by parameters is prepared, wherein the speech information is obtained based on a speech, the speaker information corresponds to the speech information, and the phonological information expresses the phoneme of the speech, and in which the parameters are determined by performing learning by sequentially inputting the speech information and the speaker information corresponding to the speech information into the probabilistic model; and
a voice conversion processing step of performing voice conversion processing of the speech information obtained on the basis of the speech of the input speaker, based both on the parameters determined in the parameter learning step and on the speaker information of a target speaker.