Systems and methods for modeling probability distributions

A non-transitory machine readable medium containing processor instructions for generating a Deep Boltzmann Machine (DBM), wherein execution of the instructions by a processor causes the processor to perform a process that comprises:

• generating values from a first restricted Boltzmann machine (RBM) comprising a visible layer and a hidden layer using a Markov chain Monte Carlo (MCMC) method by performing at least one iteration comprising:
• generating, from a first set of visible values from the visible layer, a set of hidden values in the hidden layer of the first RBM;
• and
• generating a second set of visible values in the visible layer of the first RBM based on the generated set of hidden values;
• computing a set of likelihood gradients based on at least one of the first set of visible values and the generated second set of visible values;
• computing, based on at least one of the set of hidden values and the second set of visible values, a set of adversarial gradients using an adversarial model, wherein the adversarial model includes an architecture and weights that are the same as that of the first RBM;
• computing a set of compound gradients based on the set of likelihood gradients and the set of adversarial gradients;
• updating the first RBM based on the set of compound gradients;
• sampling the hidden layer from the first RBM;
• stacking the visible layer and the hidden layer from the first RBM into a vector;
• training a second RBM having a visible layer and a hidden layer, wherein the vector is the visible layer of the second RBM and the hidden layer of the second RBM is equivalent to a layer between the visible layer and the hidden layer of the first RBM;
• and
• generating the DBM by copying weights from the first and second RBMs to the DBM, wherein the DBM comprises a visible layer and a plurality of hidden layers.