# US 2020/0401916 A1 · Systems and methods for training generative machine learning models

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0401916 A1
- Document ID: US20200401916A1
- Status: Published application
- Kind: A1
- Application number: 16/968,465
- Filed: 2019-02-07
- Published: 2020-12-24
- Earliest claimed: 2018-02-09
- Assignees: D-Wave Systems Inc.
- Applicants: D-Wave Systems Inc.
- Inventors: Jason T. Rolfe; Amir H. Khoshaman; Arash Vahdat; Mohammad H. Amin; Evgeny A. Andriyash; William G. Macready
- Web page: https://patentgrep.com/patent/US20200401916A1

## CPC

- G06N 3/088: Non-supervised learning, e.g. competitive learning
- G06N 5/04: Inference or reasoning models
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/045: Combinations of networks
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/047: Probabilistic or stochastic networks
- G06N 3/0475: Generative networks
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06N 10/60: Quantum algorithms, e.g. based on quantum optimisation, quantum Fourier or Hadamard transforms
- G06N 20/00: Machine learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Generative and inference machine learning models with discrete-variable latent spaces are provided. Discrete variables may be transformed by a smoothing transformation with overlapping conditional distributions or made natively reparametrizable by definition over a GUMBEL distribution. Models may be trained by sampling from different models in the positive and negative phase and/or sample with different frequency in the positive and negative phase. Machine learning models may be defined over high-dimensional quantum statistical systems near a phase transition to take advantage of long-range correlations. Machine learning models may be defined over graph-representable input spaces and use multiple spanning trees to form latent representations. Machine learning models may be relaxed via continuous proxies to support a greater range of training techniques, such as importance weighting. Example architectures for (discrete) variational autoencoders using such techniques are also provided. Techniques for improving training efficacy and sparsity of variational autoencoders are also provided.

## Claims

### Claim 1

A method for unsupervised learning over an input space comprising discrete or continuous variables, and at least a subset of a training dataset of samples of the discrete or continuous variables, to attempt to identify a value of at least one parameter that increases a log-likelihood of the at least a subset of a training dataset with respect to a model, the model expressible as a function of the at least one parameter, the method executed by circuitry including at least one processor and comprising;
forming a latent space comprising a plurality of random variables, the plurality of random variables comprising one or more discrete random variables and a set of supplementary continuous random variables corresponding to at least a subset of the plurality of random variables;
forming a first transforming distribution comprising a conditional distribution over the set of supplementary continuous random variables, conditioned on the one or more discrete random variables of the latent space, the first transforming distribution comprising a first smoothing distribution conditional on a first discrete value of the one or more discrete random variables and a second smoothing distribution conditional on a second discrete value of the one or more discrete random variables, the first and second smoothing distributions having the same support;
forming an encoding distribution comprising an approximating posterior distribution over the latent space, conditioned on the input space;
forming a prior distribution over the latent space;
forming a decoding distribution comprising a conditional distribution over the input space conditioned on the set of supplementary continuous random variables; and
training the model based on the first transforming distribution.

### Claim 2 (depends on 1)

The method according to claim 1 wherein training the model based on the first transforming distribution comprises:
determining an ordered set of conditional cumulative distribution functions of the supplementary continuous random variables, each cumulative distribution function comprising functions of a full distribution of at least one of the one or more discrete random variables of the latent space;
determining an inversion of the ordered set of conditional cumulative distribution functions of the supplementary continuous random variables;
constructing a first stochastic approximation to a lower bound on the log-likelihood of the at least a subset of a training dataset;
constructing a second stochastic approximation to a gradient of the lower bound on the log-likelihood of the at least a subset of a training dataset; and
increasing the lower bound on the log-likelihood of the at least a subset of a training dataset based at least in part on the gradient of the lower bound on the log-likelihood of the at least a subset of a training dataset.

### Claim 3 (depends on 1)

The method according to claim 1 wherein the first and second smoothing distributions are one of continuous and symmetric.

### Claim 4

(canceled)

### Claim 5 (depends on 1)

The method according to claim 1 wherein each of the first and second smoothing distributions is selected from the group consisting of an exponential distribution, a normal distribution, and a logistic distribution.

### Claim 6 (depends on 1)

The method according to claim 1 wherein forming a first transforming distribution comprises forming a first transforming distribution based on three or more smoothing distributions, each smoothing distribution converging to a mode of a distribution of the discrete random variables.

### Claim 7 (depends on 1)

The method according to claim 1 wherein training the model comprises optimizing an objective function based on importance sampling to determine terms associated with the first transforming distribution.

### Claim 8 (depends on 1)

The method according to claim 1 wherein forming a latent space comprises representing at least a portion of the latent space as a Boltzmann machine and wherein training the model comprises instructing a quantum processor to physically encode a quantum distribution approximating a Boltzmann distribution, instructing the quantum processor to sample from the quantum distribution, and receiving, from the quantum processor, one or more samples from the quantum distribution.

### Claim 9 (depends on 1)

The method according to claim 1 wherein training the model comprises:
optimizing an objective function having a plurality of terms; and
scaling each term of a subset of the plurality of terms of the objective function by a corresponding scaling factor, each scaling factor proportional to a magnitude of a corresponding term of the objective function.

### Claim 10 (depends on 9)

The method according to claim 9 wherein each magnitude of each corresponding term of the objective function was determined in a previous iteration of a parameter-update operation.

### Claim 11 (depends on 9)

The method according to claim 9 wherein training the model comprises:
scaling each term of the subset by a common annealing factor, the common annealing factor annealing from an initial value to 1 during training, and
removing the scaling by the common annealing factor when the common annealing factor reaches 1.

### Claim at position 12

-83. (canceled)

### Claim 84 (depends on 1)

The method according to claim 1 wherein:
forming the encoding distribution comprising the approximating posterior distribution comprises forming the approximating posterior distribution over the set of supplementary continuous random variables conditioned on the input space; and
training the model comprises determining a gradient over one or more samples from the set of supplementary continuous random variables based on a cumulative density function of the first transforming distribution.

### Claim 85 (depends on 84)

The method according to claim 84 wherein determining the gradient comprises one or more of determining the gradient with respect to a first probability yielded by the approximating posterior distribution for a first binary state of the one or more discrete random variables and determining a first value of the cumulative density function conditioned on the first binary state of the one or more discrete random variables and a second value of the cumulative density function conditioned on a second binary state of the one or more discrete random variables.

### Claim at position 15

-89. (canceled)

### Claim 90 (depends on 1)

The method according to claim 1 wherein forming the first transforming distribution comprises determining the first smoothing distribution so as to remove one or more pairwise interactions between the one or more discrete random variables.

### Claim 91 (depends on 90)

The method according to claim 90 wherein determining the first smoothing distribution comprises determining the first smoothing distribution based on a Gaussian distribution with variance based on the one or more pairwise interactions between the one or more discrete random variables.

### Claim 92 (depends on 91)

The method according to claim 91 wherein determining the Gaussian distribution comprises determining a modified interaction matrix based on the one or more pairwise interactions and a modifying term.

### Claim at position 19

-99. (canceled)

### Claim 100 (depends on 1)

The method according to claim 1 wherein:
forming the first transforming distribution comprises forming the first smoothing distribution and the second smoothing distribution factorially; and
training the model comprises:
determining a modified model based on the first transforming distribution and
approximating at least a portion of an objective function over the set of supplementary continuous random variables based on a mean-field approximation of the modified model.

### Claim 101 (depends on 100)

The method according to claim 100 wherein the modified model comprises a distribution over the set of supplementary continuous random variables and the approximating at least a portion of the objective function comprises fitting the mean-field approximation of the modified model by minimizing a difference metric between the mean-field approximation and the modified model.

### Claim 102 (depends on 101)

The method according to claim 101 wherein minimizing the difference metric comprises minimizing a Kullback-Leibler divergence.

### Claim 103 (depends on 100)

The method according to claim 100 wherein:
the model comprises a Boltzmann machine;
determining the modified model comprises determining a modified Boltzmann machine based on the first transforming distribution; and
training the model comprises:
determining a first gradient over a first energy term corresponding to the modified Boltzmann machine; and
determining an expectation of a second gradient over a second energy term corresponding to the Boltzmann machine.

### Claim at position 24

-111. (canceled)

### Claim 112

A computational system, comprising:
at least one processor; and
at least one nontransitory processor-readable storage medium that stores at least one of processor-executable instructions or data which, when executed by the at least one processor, cause the at least one processor to:
form a latent space comprising a plurality of random variables, the plurality of random variables comprising one or more discrete random variables and a set of supplementary continuous random variables corresponding to at least a subset of the plurality of random variables;
form a first transforming distribution comprising a conditional distribution over the set of supplementary continuous random variables, conditioned on the one or more discrete random variables of the latent space, the first transforming distribution comprising a first smoothing distribution conditional on a first discrete value of the one or more discrete random variables and a second smoothing distribution conditional on a second discrete value of the one or more discrete random variables, the first and second smoothing distributions having the same support;
form an encoding distribution comprising an approximating posterior distribution over the latent space, conditioned on an input space comprising discrete or continuous variables;
form a prior distribution over the latent space;
form a decoding distribution comprising a conditional distribution over the input space conditioned on the set of supplementary continuous random variables; and
train a model based on the first transforming distribution.