Variational inference and learning for segmental switching state space models of hidden speech dynamics

A method that facilitates modeling speech dynamics in a speech recognition system comprising:

• decoding an unobserved phone sequence of speech from acoustic data based, at least in part, upon a speech model, the speech model based upon a hidden dynamic model in the form of a segmental switching state space model, comprising one or more states corresponding to respective phones in the unobserved phone sequence having respective durations corresponding to estimated soft boundaries for the phones, and further comprising at least two sets of parameters, a first set of model parameters describing unobserved speech dynamics and a second set of model parameters describing a relationship between an unobserved speech dynamic vector and an observed acoustic feature vector;
• calculating a posterior distribution based on at least the first set of model parameters and the second set of model parameters;
• and,
• modifying at least one of the model parameters based, at least in part, upon the calculated posterior distribution.

A method of modeling speech dynamics from acoustic data for speech recognition comprising:

• recovering a phone sequence of speech from acoustic data based, at least in part, upon a speech model, wherein the speech model is a segmental switching state space model and comprises a plurality of model parameters and one or more states corresponding to respective phones in the phone sequence created by segmenting the speech model in time based on estimated soft boundaries for the phones;
• calculating an approximation of a posterior distribution based on the model parameters, the model parameters and the approximation based upon a mixture of Gaussians;
• and,
• modifying at least one model parameter based, at least in part, upon the calculated approximated posterior distribution and minimization of a Kullback-Leibler distance of the approximation from an exact posterior distribution.

A method that facilitates creating a model of speech dynamics for a speech recognition application comprising:

• recovering a phone sequence of speech from acoustic data based, at least in part, upon a speech model in the form of a segmental switching state space model comprising one or more states respectively corresponding to the phone sequence, the states are generated by segmenting the speech model in time based on soft boundaries for respective phones in the phone sequence;
• calculating an approximation of a posterior distribution based on model parameters, the model parameters and the approximation based upon a hidden Markov model posterior;
• and,
• modifying at least one of the model parameters based, at least in part, upon the calculated approximated posterior distribution and minimization of a Kullback-Leibler distance of the approximation from an exact posterior distribution.

A computer readable medium containing computer executable instructions operable to perform a method of modeling speech dynamics comprising:

• receiving acoustic data;
• modeling speech based on a segmental switching state space model comprising a first set of parameters that describe unobserved speech dynamics, a second set of parameters that describe a relationship between the unobserved speech dynamic vector and an observed acoustic feature vector, and a set of states having respective durations corresponding to soft phone boundaries determined from the acoustic data;
• and
• modifying at least one of the first set of parameters and the second set of parameters based, at least in part, upon a variational learning technique.

A system that facilitates modeling speech dynamics comprising:

• means for receiving acoustic data;
• and,
• means for characterizing speech as a segmental switching state space model based, at least in part, upon the acoustic data,
• wherein the means for modeling speech employs model parameters that are modified based, at least in part, upon a variational learning technique and one or more states having respective durations corresponding to estimated soft phone boundaries.

A method of modeling speech dynamics for a speech processing application, comprising:

• constructing a speech model, the speech model is based on a hidden dynamic model in the form of a segmental switching state space model for speech applications, the constructing a speech model comprising:
• initializing a first set of model parameters that describes unobserved vocal tract resonance frequencies;
• initializing a second set of model parameters that describes a mapping relationship between the unobserved vocal tract resonance frequencies and observed acoustic data;
• creating a state equation based on the first set of model parameters to express the unobserved vocal tract resonance frequencies as a set of states respectively corresponding to phones in an unobserved phonetic transcript, the state equation is a linear dynamic equation that describes transitions between states in the set of states in terms of a phone-dependent system matrix and a target vector and includes a first Gaussian noise parameter;
• creating an observation equation that utilizes the first set of model parameters and the second set of model parameters to represent a phone-dependent mapping between the unobserved vocal tract resonance frequencies and the observed acoustic data, the mapping selected from the group consisting of a linear mapping and a piecewise linear mapping within respective phones, the observation equation includes a second Gaussian noise parameter;
• estimating soft phone boundaries for phones in the unobserved phonetic transcript under an expectation-maximization (EM) framework;
• and
• constructing a series of time-varying transition matrices based on the phonetic transcript to constrain the set of states to respective time durations corresponding to the estimated soft phone boundaries for phones in the phonetic transcript, thereby forcing the states to be consistent in time with the phonetic transcript;
• calculating an estimated multimodal posterior distribution based on the constructed speech model, the first set of model parameters, and the second set of model parameters;
• and
• modifying one or more model parameters to minimize a Kullback-Leibler distance from the estimated multimodal posterior distribution to an exact posterior distribution, the modifying is based on an EM framework having an expectation step of model inference and a maximization step of model learning, the model learning is based on a variational learning technique that employs calculus of variation.