Methods and systems for reducing dimensionality in a reduction and prediction framework

A data processing system comprising one or more processors and one or more non-transitory storage mediums storing software instructions, that, when executed by the one or more processors cause the data processing system to perform a method comprising:

• receiving an sensed data point from an industrial process;
• applying a mapping model to map the sensed data point to a respective embedding that has reduced dimensionality relative to the sensed data point;
• determining, based on a comparison of the respective embedding to prior embeddings, if the mapping model needs to be updated or not:
• when the mapping model needs to be updated, applying manifold learning to learn an updated set of model parameters for the mapping model;
• and
• when the mapping model does not need to be updated, applying a classification model to the respective embedding to predict a classification for the sensed data point.

A method comprising:

• determining if a mapping model is accurate for mapping out-of-sample (OoS) data points to respective embeddings;
• if the mapping model is determined to be accurate, using the mapping model to map the OoS data points to the respective embeddings and a second model to map the embeddings to a prediction;
• and
• if the mapping model is determined not to be accurate:
• determining new mapping model parameters for the mapping model using manifold learning;
• determining new second model parameters for the second model;
• using the mapping model with the updated mapping model parameters to map the OoS data points to new respective embeddings and using the second model with the updated second model parameters to map the new respective embeddings to a prediction.