Time series prediction with confidence estimates using sparse recurrent mixture density networks

A system for time-series prediction using a sparse recurrent mixture density networks (RMDN) model, comprising:

• one or more memories;
• and
• one or more first hardware processors, the one or more first memories coupled to the one or more first hardware processors, wherein the one or more first hardware processors are configured to execute programmed instructions stored in the one or more first memories to:
• iteratively predict time series in a plurality of iterations using a data set comprising a plurality of high-dimensional time series, the plurality of high-dimensional time series comprising a first set of high-dimensional time series associated with a training data and a second set of the high-dimensional time series associated with a validation data, wherein in each iteration of the plurality of iterations, the one or more hardware processors are configured by the instructions to:
• pass, through a feedforward layer of the sparse RMDN model, a high-dimensional time series from amongst the plurality of high-dimensional time series, the sparse RMDN model comprising the feedforward layer, a recurrent neural network (RNN) and a mixture density network (MDN), the feedforward layer comprising a plurality of units associated with a plurality of distinct weights learnt by training the sparse RMDN model, the sparse RMDN model being trained by imposing Lasso penalty on the plurality of weights of the feedforward layer to determine a set of features associated with the time series in an unsupervised manner;
• perform, by the feedforward layer, dimensionality reduction of the high-dimensional time series to obtain a reduced dimensional time series, the feedforward layer comprising a number of the plurality of units equal to a fraction of the number of features in the set of features to perform the dimensionality reduction;
• feed, through the RNN, the reduced dimensional time series to obtain latent representation of the high-dimensional time-series, the latent representation captures temporal patterns from the reduced dimensional time series;
• feed the latent representation of the high-dimensional time series to a mixture of Gaussians comprising a plurality of Gaussian components to predict a plurality of parameters associated with the plurality of Gaussian components in the mixture, the plurality of parameters comprising a plurality of probability values, a plurality of mean values and a plurality of standard deviation values associated with the plurality of the Gaussian components;
• select a Gaussian component from amongst the plurality of Gaussian components that is associated with a highest value of probability from amongst the plurality of probability values, wherein the mean of the selected Gaussian component is selected as prediction of the time-series and the standard deviation of the Gaussian component is selected for confidence estimation of the prediction for the iteration;
• compute a value of a loss function indicative of error in the prediction of the time-series using the plurality of parameters, the loss function being one of a training loss function and a validation loss function;
• and
• update a plurality of weights of the sparse RMDN model using the value of the training loss function after each iteration of the plurality of iterations associated with the training data for prediction of the time-series.

One or more non-transitory machine readable information storage mediums comprising one or more instructions which when executed by one or more hardware processors cause:

• iteratively predicting, via one or more hardware processors, time series in a plurality of iterations using a data set comprising a plurality of high-dimensional time series, the plurality of high-dimensional time series comprising a first set of high-dimensional time series associated with a training data and a second set of the high-dimensional time series associated with a validation data, each iteration of the plurality of iterations comprising:
• passing, through a feedforward layer of the sparse RMDN model, a high-dimensional time series from amongst the plurality of high-dimensional time series, the sparse RMDN model comprising the feedforward layer, a recurrent neural network (RNN) and a mixture density network (MDN), the feedforward layer comprising a plurality of units associated with a plurality of distinct weights learnt by training the sparse RMDN model, the sparse RMDN model being trained by imposing Lasso penalty on the plurality of weights of the feedforward layer to determine a set of features associated with the time series in an unsupervised manner;
• performing, by the feedforward layer, dimensionality reduction of the high-dimensional time series to obtain a reduced dimensional time series, the feedforward layer comprising a number of the plurality of units equal to a fraction of the number of features in the set of features to perform the dimensionality reduction;
• feeding, through the RNN, the reduced dimensional time series to obtain latent representation of the high-dimensional time-series, the latent representation captures temporal patterns from the reduced dimensional time series;
• feeding the latent representation of the high-dimensional time series to a mixture of Gaussians comprising a plurality of Gaussian components to predict a plurality of parameters associated with the plurality of Gaussian components in the mixture, the plurality of parameters comprising a plurality of probability values, a plurality of mean values and a plurality of standard deviation values associated with the plurality of the Gaussian components;
• selecting a Gaussian component from amongst the plurality of Gaussian components that is associated with a highest value of probability from amongst the plurality of probability values, wherein the mean of the selected Gaussian component is selected as prediction of the time-series and the standard deviation of the Gaussian component is selected for confidence estimation of the prediction for the iteration;
• computing a value of a loss function indicative of error in the prediction of the time-series using the plurality of parameters, the loss function being one of a training loss function and a validation loss function;
• and
• updating, via the one or more hardware processors, a plurality of weights of the sparse RMDN model using the value of the training loss function after each iteration of the plurality of iterations associated with the training data for prediction of the time-series.