Anomaly detection and anomalous patterns identification

A computer program product for end-to-end anomaly detection and anomalous patterns identification, the computer program product comprising:

• one or more computer readable storage media and program instructions stored on the one or more computer readable storage media, the program instructions comprising the steps of:
• receiving multivariate time-series metrics data collected from one or more computing systems over time;
• segmenting the multivariate time-series metrics data into time-bounded windows, each window comprising a plurality of timestamped metric values;
• classifying, by a multivariate time-series classification model trained using labeled ground-truth windows, each time-bounded window as normal or anomalous;
• aggregating metric values within each anomalous window into abnormal feature vectors by computing, for each metric dimension, one or more statistical features comprising at least one of a mean value, a maximum value, or a minimum value;
• initializing a plurality of anomalous pattern clusters by applying a clustering initialization technique to the abnormal feature vectors, each cluster defined by probabilistic parameters comprising a mean and a variance;
• iteratively determining cluster membership probabilities for the abnormal feature vectors using a Gaussian mixture model (GMM) by executing (i) an expectation step that computes, for each abnormal feature vector of the abnormal feature vectors, a probability of membership in each cluster based on the probabilistic parameters and (ii) a maximization step that updates the probabilistic parameters using the determined cluster membership probabilities;
• performing feature selection during the iterative clustering initialization technique using a LASSO-based regression technique that incorporates feature values from the abnormal feature vectors and normal feature vectors to identify metric dimensions contributing to each anomalous pattern cluster associated with the plurality of anomalous pattern clusters;
• and
• terminating the iterative clustering initialization technique in response to a convergence condition being satisfied by comparing variation of the probabilistic parameters across iterations to a predefined threshold, and outputting the plurality of anomalous pattern clusters and associated critical features characterizing distinct anomalous behaviors.

A computer system for end-to-end anomaly detection and anomalous patterns identification, the computer system comprising:

• one or more computer processors;
• one or more computer readable storage media;
• and
• program instructions stored on the one or more computer readable storage media for execution by at least one of the one or more computer processors, the program instructions comprising the steps of:
• receiving multivariate time-series metrics data collected from one or more computing systems over time;
• segmenting the multivariate time-series metrics data into time-bounded windows, each window comprising a plurality of timestamped metric values;
• classifying, by a multivariate time-series classification model trained using labeled ground-truth windows, each time-bounded window as normal or anomalous;
• aggregating metric values within each anomalous window into abnormal feature vectors by computing, for each metric dimension, one or more statistical features comprising at least one of a mean value, a maximum value, or a minimum value;
• initializing a plurality of anomalous pattern clusters by applying a clustering initialization technique to the abnormal feature vectors, each cluster defined by probabilistic parameters comprising a mean and a variance;
• iteratively determining cluster membership probabilities for the abnormal feature vectors using a Gaussian mixture model (GMM) by executing (i) an expectation step that computes, for each abnormal feature vector of the abnormal feature vectors, a probability of membership in each cluster based on the probabilistic parameters and (ii) a maximization step that updates the probabilistic parameters using the determined cluster membership probabilities;
• performing feature selection during the iterative clustering initialization technique using a LASSO-based regression technique that incorporates feature values from the abnormal feature vectors and normal feature vectors to identify metric dimensions contributing to each anomalous pattern cluster associated with the plurality of anomalous pattern clusters;
• and
• terminating the iterative clustering initialization technique in response to a convergence condition being satisfied by comparing variation of the probabilistic parameters across iterations to a predefined threshold, and outputting a plurality of anomalous pattern clusters and associated critical features characterizing distinct anomalous behaviors.