# US 2021/0241754 A1 · Pronunciation conversion apparatus, pitch mark timing extraction apparatus, methods and programs for the same

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0241754 A1
- Document ID: US20210241754A1
- Status: Published application
- Kind: A1
- Application number: 17/049,554
- Filed: 2019-04-09
- Published: 2021-08-05
- Earliest claimed: 2018-04-25
- Assignees: Nippon Telegraph and Telephone Corporation
- Applicants: Nippon Telegraph and Telephone Corporation
- Inventors: Sadao HIROYA
- Web page: https://patentgrep.com/patent/US20210241754A1

## CPC

- G10L 13/0335: {Pitch control}
- G10L 15/187: Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
- G09B 19/06: Foreign languages (with audible presentation of material to be studied G09B5/04)
- G10L 15/02: Feature extraction for speech recognition; Selection of recognition unit
- G10L 15/04: Segmentation; Word boundary detection
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L 19/06: Determination or coding of the spectral characteristics, e.g. of the short-term prediction coefficients
- G10L 21/007: characterised by the process used
- G10L 25/06: the extracted parameters being correlation coefficients
- G10L 25/12: the extracted parameters being prediction coefficients
- G10L 25/18: the extracted parameters being spectral information of each sub-band
- G10L 25/90: Pitch determination of speech signals
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B)

## Abstract

Provided is a system which allows a learner who is a non-native speaker of a given language to intuitively improve pronunciation of the language. A pronunciation conversion apparatus includes a conversion section which converts a first feature value corresponding to a first speech signal obtained when a first speaker who speaks a given language as his/her native language speaks another language such that the first feature value approaches a second feature value corresponding to a second speech signal obtained when a second speaker who speaks the other language as his/her native language speaks the other language, each of the first feature value and the second feature value is a feature value capable of representing a difference in pronunciation, and a speech signal obtained from the first feature value after the conversion is presented to the first speaker.

## Claims

### Claim 1

A pronunciation conversion apparatus comprising:
a conversion section which converts a first feature value corresponding to a first speech signal obtained when a first speaker who speaks a given language as his/her native language speaks another language such that the first feature value approaches a second feature value corresponding to a second speech signal obtained when a second speaker who speaks the other language as his/her native language speaks the other language, wherein
each of the first feature value and the second feature value is a feature value capable of representing a difference in pronunciation, and
a speech signal obtained from the first feature value after the conversion is presented to the first speaker.

### Claim 2 (depends on 1)

The pronunciation conversion apparatus according to claim 1, including:
a speech segment detection section which detects a speech segment of the first speech signal;
an LPC analysis section which determines an LPC residual signal by using an LPC coefficient a obtained from the first speech signal of the speech segment through LPC analysis by using the first speech signal and the speech segment, and the first speech signal of the speech segment;
a pitch mark analysis section which calculates an impulse train corresponding to an EGG glottal closure time by using the first speech signal;
a cross-correlation estimation section which estimates a delay of the LPC residual signal relative to the EGG glottal closure time by estimating a cross-correlation between the LPC residual signal and the impulse train;
a pitch mark time estimation section which estimates a pitch mark time based on the delay, the impulse train, and the LPC residual signal, and
an extraction section which extracts the first feature value from the pitch mark time and the first speech signal.

### Claim 3 (depends on 1)

The pronunciation conversion apparatus according to claim 1, wherein
let z denote a root of a prediction polynomial obtained with a linear prediction coefficient, and the conversion section determines a conversion filter
[
Formula
⁢
⁢
16
]
F
⁡
(
z
)
=
1
A
′
⁡
(
z
)
1
A
⁡
(
z
)
from a first vocal tract spectrum A(z) corresponding to the first feature value and a second vocal tract spectrum A′(z) corresponding to the second feature value, and converts the first feature value by using the conversion filter F(z).

### Claim 4 (depends on 3)

The pronunciation conversion apparatus according to claim 3, wherein
let rp denote a magnitude of an imaginary root of the prediction polynomial obtained with the linear prediction coefficient determined from the first speech signal, let θp denote an angle of the imaginary root of the prediction polynomial obtained with the linear prediction coefficient, the first vocal tract spectrum A(z) is expressed as
[
Formula
⁢
⁢
17
]
A
⁡
(
z
)
=
∏
p
=
1
P
⁢
(
1
-
c
p
⁢
z
-
1
)
⁢
(
1
-
c
p
*
⁢
z
-
1
)
⁢
⁢
c
p
=
r
p
⁢
cos
⁢
θ
p
+
j
⁢
r
p
⁢
sin
⁢
θ
p
⁢
⁢
c
p
*
=
r
p
⁢
cos
⁢
θ
p
-
j
⁢
r
p
⁢
sin
⁢
θ
p
,
the second vocal tract spectrum A′(z) is expressed as
[
Formula
⁢
⁢
18
]
A
′
⁡
(
z
)
=
∏
p
=
1
P
⁢
(
1
-
c
p
′
⁢
z
-
1
)
⁢
(
1
-
c
p
′
*
⁢
z
-
1
)
,
and
[
Formula
⁢
⁢
19
]
c
p
′
=
r
p
⁢
cos
⁢
⁢
θ
p
′
+
j
⁢
r
p
⁢
sin
⁢
⁢
θ
p
′
⁢
⁢
c
p
′
*
=
r
p
⁢
cos
⁢
⁢
θ
p
′
-
j
⁢
r
p
⁢
sin
⁢
⁢
θ
p
′
⁢
⁢
or
[
Formula
⁢
⁢
20
]
c
p
′
=
r
p
′
⁢
cos
⁢
⁢
θ
p
+
jr
p
′
⁢
sin
⁢
⁢
θ
p
⁢
⁢
c
p
′
*
=
r
p
′
⁢
cos
⁢
⁢
θ
p
-
jr
p
′
⁢
sin
⁢
⁢
θ
p
⁢
⁢
or
[
Formula
⁢
⁢
21
]
c
p
′
=
r
p
′
⁢
cos
⁢
⁢
θ
p
′
+
jr
p
′
⁢
sin
⁢
⁢
θ
p
′
⁢
⁢
c
p
′
*
=
r
p
′
⁢
cos
⁢
⁢
θ
p
′
-
jr
p
′
⁢
sin
⁢
⁢
θ
p
′
is satisfied, let F′p denote a formant frequency of the second speech signal, let fs denote a sampling frequency, let B′p denote a bandwidth of the formant frequency of the second speech signal, and
[
Formula
⁢
⁢
22
]
θ
p
′
=
2
⁢
π
⁢
⁢
F
p
′
f
s
[
Formula
⁢
⁢
23
]
r
p
′
=
e
-
π
⁢
⁢
B
p
′
f
s
are satisfied.

### Claim 5 (depends on 1)

The pronunciation conversion apparatus according to claim 1, wherein
the conversion section gives a random number to the second feature value to convert the first feature value such that the first feature value approaches the second feature value to which the random number is given.

### Claim 6

A pitch mark timing extraction apparatus comprising:
a speech segment detection section which detects a speech segment of a speech signal;
an LPC analysis section which determines an LPC residual signal by using an LPC coefficient a obtained from the speech signal of the speech segment through LPC analysis by using the speech signal and the speech segment, and the first speech signal of the speech segment;
a pitch mark analysis section which calculates an impulse train corresponding to an EGG glottal closure time by using the speech signal;
a cross-correlation estimation section which estimates a delay of the LPC residual signal relative to the EGG glottal closure time by estimating a cross-correlation between the LPC residual signal and the impulse train, and
a pitch mark time estimation section which estimates a pitch mark time based on the delay, the impulse train, and the LPC residual signal.

### Claim 7

A pronunciation conversion method comprising:
a conversion step of causing a conversion section to convert a first feature value corresponding to a first speech signal obtained when a first speaker who speaks a given language as his/her native language speaks another language such that the first feature value approaches a second feature value corresponding to a second speech signal obtained when a second speaker who speaks the other language as his/her native language speaks the other language, wherein
each of the first feature value and the second feature value is a feature value capable of representing a difference in pronunciation, and
a speech signal obtained from the first feature value after the conversion is presented to the first speaker.

### Claim 8

A pitch mark timing extraction method comprising:
a speech segment detection step of causing a speech segment detection section to detect a speech segment of a speech signal;
an LPC analysis step of causing an LPC analysis section to determine an LPC residual signal by using an LPC coefficient a obtained from the speech signal of the speech segment through LPC analysis by using the speech signal and the speech segment, and the first speech signal of the speech segment;
a pitch mark analysis step of causing a pitch mark analysis section to calculate an impulse train corresponding to an EGG glottal closure time by using the speech signal;
a cross-correlation estimation step of causing a cross-correlation estimation section to estimate a delay of the LPC residual signal relative to the EGG glottal closure time by estimating a cross-correlation between the LPC residual signal and the impulse train, and
a pitch mark time estimation step of causing a pitch mark time estimation section to estimate a pitch mark time based on the delay, the impulse train, and the LPC residual signal.

### Claim 9 (depends on 1)

A non-transitory computer-readable recording medium that records a program for causing a computer to function as the pronunciation conversion apparatus according to claim 1.

### Claim 10 (depends on 6)

A non-transitory computer-readable recording medium that records a program for causing a computer to function as the pitch mark timing extraction apparatus according to claim 6.
