Systems and methods for training generative machine learning models with sparse latent spaces
AssigneeD-Wave Systems Inc.
InventorsJason T. Rolfe, Seyed Ali Saberali, William G. Macready
Life of this patent
Estimating the term…
Abstract
Generative machine learning models, such as variational autoencoders, with comparatively sparse latent spaces are provided. Continuous latent variables are activated and/or inactivated based on a state of the latent space. Activation may be controlled by corresponding binary latent variables and/or by rectification of probability distributions defined over the latent space. Sparsification may be supported by normalization of terms, such as providing an L1 or L2 prior.
Claims16
A method for unsupervised learning over an input space comprising a plurality of input variables, and at least a subset of a training dataset of samples of the respective variables, to attempt to identify the value of at least one parameter that increases the log-likelihood of the at least a subset of a training dataset with respect to a model, the model expressible as a function of the at least one parameter, the method executed by circuitry including at least one processor and comprising;
- forming a latent space comprising a plurality of continuous random latent variables;
- forming an approximating posterior distribution over the latent space, conditioned on the input space, and formed by, for each of the continuous random latent variables, truncating a corresponding encoding base distribution based on input data from the input space;
- forming a prior distribution over the latent space;
- forming a decoding distribution over the input space;
- and
- training the model based on the encoding, prior, and decoding distributions.
-26. (canceled)
A computational system, comprising:
- at least one processor;
- and
- at least one nontransitory processor-readable storage medium that stores at least one of processor-executable instructions or data which, when executed by the at least one processor cause the at least one processor to:
- form a latent space comprising a plurality of continuous random latent variables;
- form an approximating posterior distribution over the latent space, conditioned on the input space, and formed by, for each of the continuous random latent variables, truncating a corresponding encoding base distribution based on input data from the input space;
- form a prior distribution over the latent space;
- form a decoding distribution over the input space;
- and
- train the model based on the encoding, prior, and decoding distributions.
Dependencies are inferred from the claim text.