US 10,650,802 B2Granted

Voice recognition method, recording medium, voice recognition device, and robot

AssigneePanasonic Intellectual Property Management Co., Ltd.

InventorsYuji Kunitake, Yusaku Ota

Life of this patent

Estimating the term…

Loading patent…

Abstract

A voice recognition method is provided that includes extracting a first speech from the sound collected with a microphone connected to a voice processing device, and calculating a recognition result for the first speech and the confidence level of the first speech. The method also includes performing a speech for a repetition request based on the calculated confidence level of the first speech, and extracting with the microphone a second speech obtained through the repetition request. The method further includes calculating a recognition result for the second speech and the confidence level of the second speech, and generating a recognition result from the recognition result for the first speech and the recognition result for the second speech, based on the confidence level of the calculated second speech.

Claims11

Dependent claims

A voice recognition method, comprising:

receiving, via a microphone, a first speech that a speaker makes intending one word, the first speech including N phonemes, where N is a natural number of 2 or more;
calculating occurrence probabilities of all kinds of phonemes for each of the N phonemes included in the first speech;
recognizing a phoneme string, in which phonemes each having the highest probability are lined in order, to be a first phoneme string corresponding to the first speech, the phonemes corresponding to the respective N phonemes from a first phoneme to an N-th phoneme included in the first speech;
calculating a first value by multiplying together occurrence probabilities that the N phonemes included in the first phoneme string have;
when the first value is smaller than a first threshold, outputting a voice to prompt the speaker to repeat the one word, via a loudspeaker;
receiving, via the microphone, a second speech that the speaker repeats intending the one word, the second speech including M phonemes, where M is a natural number of 2 or more;
calculating occurrence probabilities of all kinds of phonemes for each of the M phonemes included in the second speech;
recognizing a phoneme string, in which phonemes each having the highest probability are lined in order, to be a second phoneme string corresponding to the second speech, the phonemes corresponding to the respective M phonemes from a first phoneme to an M-th phoneme included in the second speech;
calculating a second value by multiplying together occurrence probabilities that the M phonemes included in the second phoneme string have;
when the second value is smaller than the first threshold, extracting a phoneme having occurrence probability higher than a second threshold out of the first phoneme string and a phoneme having occurrence probability higher than the second threshold out of the second phoneme string;
extracting a word including the extracted phonemes from a dictionary stored in a memory, the dictionary associating words with respective phoneme strings;
and
when the number of extracted words is one, recognizing the extracted word to be the one word.

10 more elements

A voice recognition method, comprising:

receiving, via a microphone a, first speech that a speaker makes intending one word string, the first speech including N phonemes, where N is a natural number of 2 or more;
calculating a confidence level X1 of a word string estimated for the first speech
X
⁢
⁢
1
=
max
⁢
∏
t
=
1
T
⁢
⁢
P
A
⁢
⁢
1
⁡
(
o
t
,
s
t
|
s
t
-
1
)
⁢
P
L
⁢
⁢
1
⁡
(
s
t
,
s
t
-
1
)
where t is a number specifying one of frames constituting the first speech,
T is the total number of the frames constituting the first speech,
PA1(ot,st|st-1) is a probability that a certain phoneme appears at a t-th frame, which is next to a phoneme string corresponding to a state st-1 of from a first frame to a (t−1)-th frame of the first speech, and the phoneme string corresponding to the state st-1 transitions to a phoneme string corresponding to a state st,
ot is a physical quantity that is for estimating the certain phoneme and is obtained from the first speech,
the certain phoneme is one of all kinds of phonemes, and
PL1(st,st-1) is a probability that a certain word appears at a t-th frame next to a word string corresponding to a state st-1, and the word string corresponding to the state st-1 transitions to a word string corresponding to a state st in the first speech;
determining whether the confidence level X1 is higher than or equal to a threshold;
when the confidence level X1 is lower than the threshold, outputting a voice to prompt the speaker to repeat the one word string, via a loudspeaker;
receiving, via the microphone, a second speech that the speaker repeats intending the one word string;
when the confidence level X1 of the second speech is lower than the threshold, calculating a combined confidence level X for each of all word strings estimated from the first speech and the second speech
X
=
∏
t
=
1
T
⁢
⁢
{
P
A
1
⁡
(
o
t
,
s
t
|
s
t
-
1
)
+
P
A
2
⁢
⁡
(
q
t
,
s
t
|
s
t
-
1
)
}
⁢
P
L
⁡
(
s
t
,
s
t
-
1
)
where t is a number specifying one of frames constituting the first speech and the second speech,
T is the total number of the frames constituting the first speech and the second speech,
PA1(ot,st|st-1) is a probability that a certain phoneme appears at a t-th frame, which is next to a phoneme string corresponding to a state st-1 of from a first frame to a (t−1)-th frame of the first speech, and the phoneme string corresponding to the state st-1 transitions to a phoneme string corresponding to a state st,
ot is a physical quantity that is for estimating the certain phoneme and is obtained from the first speech,
the certain phoneme is one of all kinds of phonemes,
PA2(qt,st|st-1) is a probability that a certain phoneme appears at a t-th frame, which is next to a phoneme string corresponding to a state st-1 of from a first frame to a (t−1)-th frame of the second speech, and the phoneme string corresponding to the state st-1 transitions to a phoneme string corresponding to a state st,
qt is a physical quantity that is for estimating the certain phoneme and is obtained from the second speech, and
PL(st,st-1) is a probability that a certain word appears at a t-th frame next to a word string corresponding to the state st-1 and the word string corresponding to the state st-1 transitions to a word string corresponding to the state st in the first speech;
and
recognizing a word string corresponding to the state st that gives the largest value of the combined confidence levels X, as the one word string.

125 more elements

A voice recognition method, comprising:

receiving, via a microphone, a first speech that a speaker makes intending one word string, the first speech including N phonemes, where N is a natural number of 2 or more;
calculating a confidence level X1 of a word string estimated for the first speech
X
⁢
⁢
1
=
∏
t
⁢
⁢
1
=
1
T
⁢
⁢
1
⁢
⁢
P
A
⁢
⁢
1
⁡
(
o
t
⁢
⁢
1
,
s
t
⁢
⁢
1
|
s
t
⁢
⁢
1
-
1
)
⁢
P
L
⁢
⁢
1
⁡
(
s
t
⁢
⁢
1
,
s
t
⁢
⁢
1
-
1
)
where t1 is a number specifying one of frames constituting the first speech,
T1 is the total number of the frames constituting the first speech,
PA1(ot1,st1|st-1) is a probability that a certain phoneme appears at a t1-th frame, which is next to a phoneme string corresponding to a state st1-1 of from a first frame to a (t1−1)-th frame of the first speech, and the phoneme string corresponding to the state st1-1 transitions to a phoneme string corresponding to a state st1,
ot1 is a physical quantity that is for estimating the certain phoneme and is obtained from the first speech,
the certain phoneme is one of all kinds of phonemes, and
PL1(st1,st1-1) is a probability that a certain word appears at the t1-th frame next to a word string corresponding to a state st1-1, and the word string corresponding to the state st1-1 transitions to a word string corresponding to a state st1 in the first speech;
determining whether the largest value MaxX1 of the confidence levels X1 is higher than or equal to a threshold;
when the largest value MaxX1 is smaller than the threshold, extracting first word strings that are estimated for the first speech and give top M values of the confidence levels X1, where M is a natural number of 2 or more;
outputting a voice to prompt the speaker to repeat the one word string, via a loudspeaker;
receiving via the microphone a second speech that the speaker repeats intending the one word string;
calculating a confidence level X2 for each one of all word strings estimated for the second speech
X
⁢
⁢
2
=
∏
t
⁢
⁢
2
=
1
T
⁢
⁢
2
⁢
⁢
P
A
⁢
⁢
2
⁡
(
o
t
⁢
⁢
2
,
s
t
⁢
⁢
2
|
s
t
⁢
⁢
2
-
1
)
⁢
P
L
⁢
⁢
2
⁡
(
s
t
⁢
⁢
2
,
s
t
⁢
⁢
2
-
1
)
where t2 is a number specifying one of frames constituting the second speech,
T2 is the total number of the frames constituting the second speech,
PA2(ot1,st2|st2-1) is a probability that a certain phoneme appears at a t2-th frame, which is next to a phoneme string corresponding to a state st2-1 of from a first frame to a (t2−1)-th frame of the second speech, and the phoneme string corresponding to the state st2-1 transitions to a phoneme string corresponding to a state st2,
ot2 is a physical quantity that is for estimating the certain phoneme and is obtained from the second speech, and
PL2(st2,st2-1) is a probability that a certain word appears at a t2-th frame next to a word string corresponding to a state st2-1, and the word string corresponding to the state st2-1 transitions to a word string corresponding to a state st2 in the second speech;
determining whether the largest value MaxX2 of the confidence levels X2 is larger than or equal to a threshold:
when the largest value MaxX2 is lower than the threshold, extracting second word strings that are estimated for the second speech and give top M values of the confidence levels X2;
and
when there is a common word string between the first word strings and the second word strings, recognizing the common word string as the one word string.

153 more elements

A voice recognition device, comprising:

a processor,
a memory,
a microphone, and
a loudspeaker,
wherein the processor
receives, via the microphone, a first speech that a speaker makes intending one word, the first speech including N phonemes, where N is a natural number of 2 or more;
calculates occurrence probabilities of all kinds of phonemes for each of the N phonemes included in the first speech;
recognizes a phoneme string, in which phonemes each having the highest probability are lined in order, to be a first phoneme string corresponding to the first speech, the phonemes corresponding to the respective N phonemes from a first phoneme to an N-th phoneme included in the first speech;
calculates a first value by multiplying together occurrence probabilities that the N phonemes included in the first phoneme string have;
when the first value is smaller than a first threshold, outputs a voice to prompt the speaker to repeat the one word, via the loudspeaker;
receives, via the microphone, a second speech that the speaker repeats intending the one word, the second speech including M phonemes, where M is a natural number of 2 or more;
calculates occurrence probabilities of all kinds of phonemes for each of the M phonemes included in the second speech;
recognizes a phoneme string in which phonemes each having the highest probability are lined in order, to be a second phoneme string corresponding to the second speech, the phonemes corresponding to the respective M phonemes from a first phoneme to an M-th phoneme included in the second speech;
calculates a second value by multiplying together occurrence probabilities that the M phonemes included in the second phoneme string have;
when the second value is smaller than the first threshold, extracts a phoneme having occurrence probability higher than a second threshold out of the first phoneme string and a phoneme having occurrence probability higher than the second threshold out of the second phoneme string;
extracts a word including the extracted phonemes from a dictionary stored in the memory, the dictionary associating words with respective phoneme strings;
and
when the number of extracted words is one, recognizes the extracted word to be the one word.

15 more elements

A voice recognition device, comprising:

a processor,
a microphone, and
a loudspeaker,
wherein the processor
receives via the microphone a first speech that a speaker makes intending one word string, the first speech including N phonemes, where N is a natural number of 2 or more;
calculates a confidence level X1 of a word string estimated for the first speech
X
⁢
⁢
1
=
max
⁢
∏
t
=
1
T
⁢
⁢
P
A
⁢
⁢
1
⁡
(
o
t
,
s
t
|
s
t
-
1
)
⁢
P
L
⁢
⁢
1
⁡
(
s
t
,
s
t
-
1
)
where t is a number specifying one of frames constituting the first speech,
T is the total number of the frames constituting the first speech,
PA1(ot,st|st-1) is a probability that a certain phoneme appears at a t-th frame, which is next to a phoneme string corresponding to a state st-1 of from a first frame to a (t−1)-th frame of the first speech, and the phoneme string corresponding to the state st-1 transitions to a phoneme string corresponding to a state st,
ot is a physical quantity that is for estimating the certain phoneme and is obtained from the first speech,
the certain phoneme is one of all kinds of phonemes, and
PL1(st,st-1) is a probability that a certain word appears at a t-th frame next to a word string corresponding to a state st-1, and the word string corresponding to the state st-1 transitions to a word string corresponding to a state st in the first speech;
determines whether the confidence level X1 is higher than or equal to a threshold;
when the confidence level X1 is lower than the threshold, outputs a voice to prompt the speaker to repeat the one word string, via the loudspeaker;
receives, via the microphone, a second speech that the speaker repeats intending the one word string;
when the confidence level X1 of the second speech is lower than the threshold, calculates a combined confidence level X for each of all word strings estimated from the first speech and the second speech
X
=
∏
t
=
1
T
⁢
⁢
{
P
A
1
⁡
(
o
t
,
s
t
|
s
t
-
1
)
+
P
A
2
⁡
(
q
t
,
s
t
|
s
t
-
1
)
}
⁢
P
L
⁡
(
s
t
,
s
t
-
1
)
where t is a number specifying one of frames constituting the first speech and the second speech,
T is the total number of the frames constituting the first speech and the second speech,
PA1(ot,st|st-1) is a probability that a certain phoneme appears at a t-th frame, which is next to a phoneme string corresponding to a state st-1 of from a first frame to a (t−1)-th frame of the first speech, and the phoneme string corresponding to the state st-1 transitions to a phoneme string corresponding to a state st,
ot is a physical quantity that is for estimating the certain phoneme and is obtained from the first speech,
the certain phoneme is one of all kinds of phonemes,
PA2(qt,st|st-1) is a probability that a certain phoneme appears at a t-th frame, which is next to a phoneme string corresponding to a state st-1 of from a first frame to a (t−1)-th frame of the second speech, and the phoneme string corresponding to the state st-1 transitions to a phoneme string corresponding to a state st,
qt is a physical quantity that is for estimating the certain phoneme and is obtained from the second speech, and
PL(st,st-1) is a probability that a certain word appears at a t-th frame next to a word string corresponding to the state st-1, and the word string corresponding to the state st-1 transitions to a word string corresponding to the state st in the first speech;
and
recognizes a word string corresponding to the state st that gives the largest value of the combined confidence levels X, as the one word string.

128 more elements

A voice recognition device, comprising:

a processor,
a microphone, and
a loudspeaker,
wherein the processor
receives, via the microphone, a first speech that a speaker makes intending one word string, the first speech including N phonemes, where N is a natural number of 2 or more;
calculates a confidence level X1 of a word string estimated for the first speech
X
⁢
⁢
1
=
∏
t
⁢
⁢
1
=
1
T
⁢
⁢
1
⁢
⁢
P
A
⁢
⁢
1
⁡
(
o
t
⁢
⁢
1
,
s
t
⁢
⁢
1
|
s
t
⁢
⁢
1
-
1
)
⁢
P
L
⁢
⁢
1
⁡
(
s
t
⁢
⁢
1
,
s
t
⁢
⁢
1
-
1
)
where t1 is a number specifying one of frames constituting the first speech,
T1 is the total number of the frames constituting the first speech,
PA1(ot1,st1|st1-1) is a probability that a certain phoneme appears at a t1-th frame which is next to a phoneme string corresponding to a state st1-1 of from a first frame to a (t1−1)-th frame of the first speech, and the phoneme string corresponding to the state st1-1 transitions to a phoneme string corresponding to a state st1,
ot1 is a physical quantity that is for estimating the certain phoneme and is obtained from the first speech,
the certain phoneme is one of all kinds of phonemes, and
PL1(st1,st1-1) is a probability that a certain word appears at the t1-th frame next to a word string corresponding to a state st1-1, and the word string corresponding to the state st1-1 transitions to a word string corresponding to a state st1 in the first speech;
determines whether the largest value MaxX1 of the confidence levels X1 is higher than or equal to a threshold;
when the largest value MaxX1 is smaller than the threshold, extracts first word strings that are estimated for the first speech and give top M values of the confidence levels X1, where M is a natural number of 2 or more;
outputs a voice to prompt the speaker to repeat the one word string, via the loudspeaker;
receives via the microphone a second speech that the speaker repeats intending the one word string;
calculates a confidence level X2 for each one of all word strings estimated for the second speech
X
⁢
⁢
2
=
∏
t
⁢
⁢
2
=
1
T
⁢
⁢
2
⁢
⁢
P
A
⁢
⁢
2
⁡
(
o
t
⁢
⁢
2
,
s
t
⁢
⁢
2
|
s
t
⁢
⁢
2
-
1
)
⁢
P
L
⁢
⁢
2
⁡
(
s
t
⁢
⁢
2
,
s
t
⁢
⁢
2
-
1
)
where t2 is a number specifying one of frames constituting the second speech,
T2 is the total number of the frames constituting the second speech,
PA2(ot2,st2|st2-1) is a probability that a certain phoneme appears at a t2-th frame, which is next to a phoneme string corresponding to a state st2-1 of from a first frame to a (t2−1)-th frame of the second speech, and the phoneme string corresponding to the state st2-1 transitions to a phoneme string corresponding to a state st2,
ot2 is a physical quantity that is for estimating the certain phoneme and is obtained from the second speech, and
PL2(st2,st2-1) is a probability that a certain word appears at a t2-th frame next to a word string corresponding to a state st2-1, and the word string corresponding to the state st2-1 transitions to a word string corresponding to a state st2 in the second speech;
determines whether the largest value MaxX2 of the confidence levels X2 is larger than or equal to a threshold:
when the largest value MaxX2 is lower than the threshold, extracts second word strings that are estimated for the second speech and give top M values of the confidence levels X2;
and
when there is a common word string between the first word strings and the second word strings, recognizes the common word string as the one word string.

157 more elements

Dependencies are inferred from the claim text.

Facts

Application: 16/019,701Filed 27 Jun 2018
Priority: JP 2017-1317625 Jul 2017
Published: US 2019/0013008 A110 Jan 2019
Granted: 12 May 2020
Term ends: Not estimatedThe source records don’t support an estimate.

Classification8 codes

G10L 15/02
Speech recognition: feature extraction for speech recognition; Selection of recognition unit
G10L 15/08
Speech recognition: speech classification or search
G10L 15/22
Speech recognition: procedures used during a speech recognition process, e.g. man-machine dialogue
G10L 15/32
Constructional details of speech recognition systems: multiple recognisers used in sequence or in parallel; Score combination systems therefor,…