Nonlinear and flexible inference of latent factors and behavior from single-modal and multi-modal brain signals

A method for at least one of inferring spatiotemporal dynamics in a plurality of brain signals or decoding behavior based on the plurality of brain signals, the method comprising:

• separating, by one or more processors and via a neural network, a plurality of latent factors from a training dataset into a set of dynamic latent factors and a set of manifold latent factors, the training dataset comprising at least one of a plurality of training brain signals or a plurality of training behavior signals;
• jointly training, by the one or more processors and via the neural network, a machine-learning model with the set of dynamic latent factors and the set of manifold latent factors to generate a trained machine-learning model;
• and
• performing, by the one or more processors and via the trained machine-learning model, at least one of:
• flexible inference of latent factors from the plurality of brain signals, or
• flexible decoding of behavior based on the plurality of brain signals.

An article of manufacture including one or more non-transitory, tangible computer readable storage mediums having instructions stored thereon that, in response to execution by one or more processors, cause the one or more processors to perform operations comprising:

• performing, by the one or more processors and via a trained machine-learning model, at least one of:
• flexible inference of latent factors on a plurality of brain signals;
• or
• flexible decoding of behavior based on the plurality of brain signals, wherein the trained machine-learning model is pre-trained by:
• separating, via a neural network, a plurality of latent factors from a training dataset into a set of dynamic latent factors and a set of manifold latent factors, the training dataset comprising at least one of a plurality of training brain signals or a plurality of training behavior signals;
• and
• jointly training, via the neural network, a machine-learning model with the set of dynamic latent factors and the set of manifold latent factors to generate the trained machine-learning model.