US 11,853,853 B1 · Providing human-interpretable explanation for model-detected anomalies

Abstract

An anomaly detection system is disclosed capable of reporting anomalous processes or hosts in a computer network using machine learning models trained using unsupervised training techniques. In embodiments, the system assigns observed processes to a set of process categories based on the file system path of the program executed by the process. The system extracts a feature vector for each process or host from the observation records and applies the machine learning models to the feature vectors to determine an outlier metric each process or host. The processes or hosts with the highest outlier metrics are reported as detected anomalies to be further examined by security analysts. In embodiments, the machine learnings models may be periodically retrained based on new observation records using unsupervised machine learning techniques. Accordingly, the system allows the models to learn from newly observed data without requiring the new data to be manually labeled by humans.

Claims20

Dependent claims

A system comprising:

one or more computing devices that implement an anomaly detection system, configured to:
execute one or more anomaly detection models on a dataset of observation records about processes executed on individual machines to determine (a) an outlier record in the dataset, and (b) an outlier score of the outlier records, wherein the one or more anomaly detection models are trained using one or more machine learning techniques;
for individual ones of a plurality of features in the outlier record:
generate, based on the outlier record, a synthetic observation record as a comparison record to use to determine an influence of the feature on the outlier score, wherein the generation retains one or more features from the outlier record in the comparison record and modifies the feature in the comparison record;
execute the one or more anomaly detection models on the comparison record to obtain another outlier score;
and
determine, based on the outlier score and the other outlier score, an influence metric indicating the influence of the feature on the outlier score;
and
output a model interpretation result of the outlier record, wherein the model interpretation result indicates a subset of the features with highest influence metrics on the outlier score.

6 more elements

A method comprising:

performing, by one or more computing devices that implement an anomaly detection system:
executing one or more anomaly detection models on a dataset of observation records about processes executed on individual machines to determine (a) an outlier record in the dataset, and (b) an outlier score of the outlier records, wherein the one or more anomaly detection models are trained using one or more machine learning techniques;
for individual ones of a plurality of features in the outlier record:
generating, based on the outlier record, a synthetic observation record as a comparison record to use to determine an influence of the feature on the outlier score, wherein the generation retains one or more features from the outlier record in the comparison record and modifies the feature in the comparison record;
executing the one or more anomaly detection models on the comparison record to obtain another outlier score;
and
determining, based on the outlier score and the other outlier score, an influence metric indicating the influence of the feature on the outlier score;
and
outputting a model interpretation result of the outlier record, wherein the model interpretation result indicates a subset of the features with highest influence metrics on the outlier score.

6 more elements

One or more non-transitory computer-accessible storage media storing program instructions that when executed on or across one or more processors implement an anomaly detection system and cause the anomaly detection system to:

execute one or more anomaly detection models on a dataset of observation records about processes executed on individual machines to determine (a) an outlier record in the dataset, and (b) an outlier score of the outlier records, wherein the one or more anomaly detection models are trained using one or more machine learning techniques;
for individual ones of a plurality of features in the outlier record:
generate, based on the outlier record, a synthetic observation record as a comparison record to use to determine an influence of the feature on the outlier score, wherein the generation retains one or more features from the outlier record in the comparison record and modifies the feature in the comparison record;
execute the one or more anomaly detection models on the comparison record to obtain another outlier score;
and
determine, based on the outlier score and the other outlier score, an influence metric indicating the influence of the feature on the outlier score;
and
output a model interpretation result of the outlier record, wherein the model interpretation result indicates a subset of the features with highest influence metrics on the outlier score.

5 more elements

Dependencies are inferred from the claim text.

Providing human-interpretable explanation for model-detected anomalies

Life of this patent

Providing human-interpretable explanation for model-detected anomalies

Life of this patent

Abstract

Claims20