Hybrid Machine Learning to Detect Anomalies

A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a computing device, causes the computing device to be specifically configured to implement a hybrid machine learning (ML) anomaly detector comprising an ensemble of unsupervised machine learning models and a semi-supervised machine learning model, and causes the hybrid ML anomaly detector to:

• execute the ensemble of unsupervised machine learning models on log data to generate, for each entry in the log data, a predicted anomaly score and corresponding anomaly classification label of the entry as to whether the entry represents an anomalous event;
• generate a partially labeled dataset based on a selected subset of entries in the log data and other unlabeled log data in the log data;
• perform, by the semi-supervised machine learning model, a similarity analysis of the unlabeled log data in the partially labeled dataset with entries in the selected subset of entries;
• and
• propagate, by the semi-supervised machine learning model, anomaly classification labels of the selected subset of entries to the other unlabeled log data based on results of the similarity analysis to thereby generate a fully labeled dataset.

An apparatus comprising:

• at least one processor;
• and
• at least one memory coupled to the at least one processor, wherein the at least one memory comprises instructions which, when executed by the at least one processor, cause the at least one processor to be specifically configured to implement a hybrid machine learning (ML) anomaly detector comprising an ensemble of unsupervised machine learning models and a semi-supervised machine learning model, and causes the hybrid ML anomaly detector to:
• execute the ensemble of unsupervised machine learning models on log data to generate, for each entry in the log data, a predicted anomaly score and corresponding anomaly classification label of the entry as to whether the entry represents an anomalous event;
• generate a partially labeled dataset based on a selected subset of entries in the log data and other unlabeled log data in the log data;
• perform, by the semi-supervised machine learning model, a similarity analysis of the unlabeled log data in the partially labeled dataset with entries in the selected subset of entries;
• and
• propagate, by the semi-supervised machine learning model, anomaly classification labels of the selected subset of entries to the other unlabeled log data based on results of the similarity analysis to thereby generate a fully labeled dataset.