Computational implementation of gaussian process models

A system comprising:

• processing circuitry comprising a plurality of processor cores;
• and
• memory circuitry storing machine-readable instructions which, when executed by the processing circuitry, cause the system to process training data comprising a set of training data items to determine parameters of a Gaussian process (GP) model for modelling the training data items, the GP model comprising a variational Gaussian process (VGP) corresponding to a GP prior conditioned and marginalized with respect to a set of randomly-distributed inducing variables, the processing comprising:
• initializing first parameters of the VGP including a positive definite matrix-valued slack parameter;
• and
• iteratively modifying the first parameters to increase or decrease an objective function comprising an expected log-likelihood for each of at least a subset of the training data items under a respective Gaussian distribution with a predictive variance depending on the slack parameter,
• wherein at each iteration modifying the first parameters comprises:
• for each training data item of the at least subset of the training data items, determining a respective gradient estimator for the expected log-likelihood using a respective one of the plurality of processor cores;
• and
• modifying the first parameters in dependence on the determined gradient estimators, wherein at an optimal value of the slack parameter:
• the slack parameter is equal to an inverse of a covariance matrix for the set of inducing variables;
• and
• the objective function corresponds to a variational lower bound of a marginal log-likelihood for a posterior distribution corresponding to the GP prior conditioned on the at least subset of the training data items.

A computer-implemented method of processing training data comprising a set of training data items to determine parameters of a Gaussian process (GP) model for modelling the training data items, the GP model comprising a variational Gaussian process (VGP) corresponding to a GP prior conditioned and marginalized with respect to a set of randomly-distributed inducing variables, the method comprising:

• initializing first parameters of the GP model, the first parameters including a positive-definite matrix-valued slack parameter;
• and
• iteratively modifying the first parameters to increase or decrease an objective function comprising an expected log-likelihood for each training data item of at least a subset of the training data items under a respective Gaussian distribution with a predictive variance depending on the slack parameter,
• wherein at each iteration modifying the first parameters comprises:
• for each training data item of the at least subset of the training data items, determining a respective gradient estimator for the expected log-likelihood;
• and
• modifying the first parameters in dependence on the determined gradient estimators,
• wherein at an optimal value of the slack parameter:
• the slack parameter is equal to an inverse of a covariance matrix for the set of inducing variables;
• and
• the objective function corresponds to a variational lower bound of a marginal log-likelihood for a posterior distribution corresponding to the GP prior conditioned on the at least subset of the training data items.

A non-transient storage medium comprising machine-readable instructions which, when executed by processing circuitry of a computer, cause the computer to process training data comprising a plurality of training data items to determine parameters of a Gaussian process (GP) model comprising a variational Gaussian process (VGP) corresponding to a GP prior conditioned and marginalized with respect to a set of randomly-distributed inducing variables, wherein said processing of the training data comprises:

• initializing first parameters of the GP model, the first parameters including a positive-definite matrix-valued slack parameter;
• and
• iteratively modifying the first parameters to increase or decrease an objective function comprising an expected log-likelihood for each training data item of at least a subset of the training data items under a respective Gaussian distribution with a predictive variance depending on the slack parameter,
• wherein at each iteration modifying the first parameters comprises:
• for each training data item of the at least subset of the training data items, determining a respective gradient estimator for the expected log-likelihood;
• and
• modifying the first parameters in dependence on the determined gradient estimators,
• wherein at an optimal value of the slack parameter:
• the slack parameter is equal to an inverse of a covariance matrix for the set of inducing variables;
• and
• the objective function corresponds to a variational lower bound of a marginal log-likelihood for a posterior distribution corresponding to the GP prior conditioned on the at least subset of the training data items.