US 11,495,213 B2Granted

Noise speed-ups in hidden markov models with applications to speech recognition

AssigneeUniversity of Southern California

InventorsKartik Audhkhasi, Osonde Osoba, Bart Kosko

Life of this patent

Estimating the term…

Loading patent…

Abstract

A learning computer system may estimate unknown parameters and states of a stochastic or uncertain system having a probability structure. The system may include a data processing system that may include a hardware processor that has a configuration that: receives data; generates random, chaotic, fuzzy, or other numerical perturbations of the data, one or more of the states, or the probability structure; estimates observed and hidden states of the stochastic or uncertain system using the data, the generated perturbations, previous states of the stochastic or uncertain system, or estimated states of the stochastic or uncertain system; and causes perturbations or independent noise to be injected into the data, the states, or the stochastic or uncertain system so as to speed up training or learning of the probability structure and of the system parameters or the states.

Claims20

Dependent claims

A learning computer system that estimates unknown parameters and states of a stochastic or uncertain system having a probability structure comprising a data processing system that includes a hardware processor that has a configuration that:

receives data;
generates perturbations or independent noise of the data, one or more of the states, or the probability structure, wherein the perturbations are random, chaotic, fuzzy, or other numerical perturbations;
estimates states of the stochastic or uncertain system using the data, perturbations, independent noise, previous states of the stochastic or uncertain system, or estimated states of the stochastic or uncertain system;
and
causes the perturbations or the independent noise to be injected into the data, the states, or the stochastic or uncertain system so as to speed up training or learning of the probability structure and of system parameters or the states;
the perturbations or independent noise satisfying the Noisy Expectation Maximization (NEM) prescriptive condition defined by equation 3:
𝔼
O
,
ℒ
,
N
⁢

Θ
*
=
[
ln
⁢
⁢
(
f
⁢
(
O
+
N
,
ℒ
|
Θ
(
n
)
)
f
⁢
(
O
,
ℒ
|
Θ
(
n
)
)
)
]
≥
0
(
3
)
wherein is an E step of an expectation-maximization iteration;
O is observed data;
is the latent variables;
N is a noise random variable;
Θ* is a converged value of Θ(n);
Θ(n) is a sequence of EM estimates;
f is a probability distribution function.

59 more elements

A non-transitory, tangible, computer-readable storage media containing a program of instructions that cause a computer system comprising a data processing system that includes a hardware processor running the program of instructions to estimate unknown parameters and states of a stochastic or uncertain system having a probability structure that:

receives data;
generates perturbations or independent noise of the data, one or more of the states, or the probability structure, wherein the perturbations are random, chaotic, fuzzy, or other numerical perturbations;
estimates states of the stochastic or uncertain system using the data, perturbations, previous states of the stochastic or uncertain system, or estimated states of the stochastic or uncertain system;
and
causes the perturbations or the independent noise to be injected into the data, the states, or the stochastic or uncertain system so as to speed up training or learning of the probability structure and of system parameters or the states;
the perturbations or independent noise satisfying the Noisy Expectation Maximization (NEM) prescriptive condition defined by equation 3:
𝔼
O
,
ℒ
,
N
⁢

Θ
*
=
[
ln
⁢
⁢
(
f
⁢
(
O
+
N
,
ℒ
|
Θ
(
n
)
)
f
⁢
(
O
,
ℒ
|
Θ
(
n
)
)
)
]
≥
0
(
3
)
wherein is an E step of an expectation-maximization iteration;
O is observed data;
is the latent variables;
N is a noise random variable;
Θ* is a converged value of Θ(n);
Θ(n) is a sequence of EM estimates;
f is a probability distribution function.

59 more elements

Dependencies are inferred from the claim text.

Facts

Application: 14/802,760Filed 17 Jul 2015 · continuation in part
Priority: 23 Jul 2012
Published: US 2016/0005399 A17 Jan 2016
Granted: 8 Nov 2022
Term ends: Not estimatedThe source records don’t support an estimate.

Classification5 codes

G10L 15/144
Hidden Markov Models [HMMs]: training of HMMs
G10L 15/063
Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice: training
G10L 25/33
Characterised by the analysis technique: using fuzzy logic
G10L 25/36
Characterised by the analysis technique: using chaos theory

Family

Application61/674,615Us provisional application · filed 23 Jul 2012
Application13/949,048Parent of this continuation-in-part · filed 23 Jul 2013
Application62/025,664Us provisional application · filed 17 Jul 2014
Application62/032,385Us provisional application · filed 1 Aug 2014
US 2016/0005399 A1Pre-grant publication · published 7 Jan 2016
US 9,390,065 B2Parent of this continuation-in-part · filed 12 Jul 2016
US 2014/0025356 A1Published application
US 11,495,213 B2This patent

From the US filing history. Foreign family members aren’t listed.