Method for detecting intrusions in an audit log

A method for detecting intrusions in an audit log, wherein the audit log includes a plurality of records of a plurality of user sessions of a plurality of users, each record of a user session of the plurality of user sessions includes a plurality of activity features and a user label of a claimed user of the plurality of users of a user session of the plurality of user sessions of said each record of the plurality of records, the method comprising:

• training a user identification model to predict probabilities that a first user session of the plurality of user sessions belongs to the claimed user of the plurality of users,
• wherein trained user sessions of trained audit logs are provided to the user identification model;
• predicting, via said user identification model that is trained, a plurality of probabilities that the first user session belongs to the claimed user of the plurality of users,
• wherein a probability of the plurality of probabilities is predicted for each combination of the claimed user and the first user session of the audit log, such that there as many probabilities as there are user to user session combinations,
• wherein said predicting the plurality of probabilities is based on the user identification model, wherein the user identification model takes as input the plurality of activity features of the plurality of user sessions;
• constructing a user group including users with similar activity features of said plurality of activity features, wherein the user group is constructed based on the plurality of probabilities that are predicted,
• wherein said constructing said user group is performed by a clustering method applied on a confusion matrix with confusion matrix rows, wherein each row of said confusion matrix rows represents each user of said plurality of users, such that said each user is represented as a vector of predicted user probabilities,
• wherein said clustering method is performed:
• with scikit-learn implementations of Density-Based Spatial Clustering of Applications with Noise (DBSCAN) or
• by hierarchical agglomerative clustering,
• wherein said users with said similar activity features are grouped together in the user group based on a similarity threshold to reduce false positives in a presence of indistinguishable users when detecting said intrusions using User Cluster Identification Based Intrusion Detection (UCIBID), such that each user session of said plurality of user sessions which cannot be attributed to an associated user is not interpreted to be anomalous;
• wherein said UCIBID is a classifier-agnostic scheme to turn supervised user identification into unsupervised intrusion ranking;
• wherein said UCIBID leverages similarities in user activities of said plurality of users,
• wherein an efficiency of UCIBID is characterized by an area under a precision-recall curve (AUPRC), such that:
• a precision of said AUPRC is defined as a number of true positives divided by the sum of the number of true positives plus a number of said false positives, and
• a recall is defined as the number of true positives divided by the sum of the number of true positives plus a number of false negatives,
• wherein said similarity threshold is automatically determined by said clustering method;
• determining an anomaly score for the first user session of the plurality of user sessions of the audit log and the claimed user of the first user session,
• wherein the claimed user belongs to the user group,
• wherein the anomaly score is determined based on a probability that the first user session belongs to the user group,
• wherein said users with said similar activity features that are grouped together in the user group are not considered anomalous,
• wherein the probability that the first user session belongs to the user group is a sum over all predictions of the plurality of probabilities that the first user session belongs to the user group,
• wherein the lower the probability that the first user session belongs to the user group, the more anomalous the first user session is;
• detecting an intrusion if the anomaly score of the first user session and the claimed user exceeds a predetermined threshold to reduce said false positives in the presence of said indistinguishable users through said clustering of the confusion matrix;
• and
• providing an unsupervised anomaly-based intrusion ranking for said plurality of user sessions in said audit log using said UCIBID.