Framework for semi-supervised learning when no labeled data is given

An apparatus, comprising:

• a probe;
• and
• at least one processor configured:
• to collecting, from the probe, communication sessions comprising data packets transmitted over a network,
• to generate a plurality of samples from the collected data packets,
• to apply multiple classifiers to the samples, thereby computing a classifier prediction for each of the samples, each of the predictions indicating whether or not a given communication session indicates malicious activity,
• to determine, based on the classifier predictions, an aggregated prediction for each of the samples,
• to compute, based on the classifier predictions and the aggregated predictions, a precision and a hit rate for each of the classifiers and a positive rate,
• to select, based on the aggregated predictions, a subset of the samples,
• to compute, using the selected subset and based on the precisions and the hit rates of the classifiers and the positive rate, a classification model comprising the classifiers,
• to apply the classification model to the samples to update the classifier predictions and the aggregate predictions for the samples,
• to repeat over multiple iterations the steps of computing the precision and the hit rate, selecting the subset, computing the classification model and applying the classification model until a halting condition is met, and
• to scan, using the classification model, additional communication sessions comprising additional data packets.

A computer software product, the product comprising a non-transitory computer-readable medium, in which program instructions are stored, which instructions, when read by a computer, cause the computer:

• to collect communication sessions comprising data packets transmitted over a network;
• to generate a plurality of samples from the collected data packets;
• to apply multiple classifiers to the samples, thereby computing a classifier prediction for each of the samples, each of the predictions indicating whether or not a given communication session indicates malicious activity;
• to determine, based on the classifier predictions, an aggregated prediction for each of the samples;
• to compute, based on the classifier predictions and the aggregated predictions, a precision and a hit rate for each of the classifiers and a positive rate;
• to select, based on the aggregated predictions, a subset of the samples;
• to compute, using the selected subset and based on the precisions and the hit rates of the classifiers and the positive rate, a classification model comprising the classifiers;
• to apply the classification model to the samples to update the classifier predictions and the aggregate predictions for the samples;
• to repeat over multiple iterations the steps of computing the precision and the hit rate, selecting the subset, computing the classification model and applying the classification model until a halting condition is met;
• and
• to scan, using the classification model, additional communication sessions comprising additional data packets.

A method, comprising:

• receiving, from a set of classifiers, respective predictions for a set of samples;
• identifying, by a processor, a subset of the samples having disagreements in their predictions;
• receiving labels for the samples in the subset;
• and
• using the identified subset of the samples and their respective labels to compute a classification model comprising the set of classifiers.

An apparatus, comprising:

• a memory;
• and
• a processor configured:
• to receive, from a set of classifiers, respective predictions for a set of samples,
• to store the respective predictions to a memory,
• to identify a subset of the samples having disagreements in their predictions,
• to receive labels for the samples in the subset, and
• to use the identified subset of the samples and their respective labels to compute a classification model comprising the set of classifiers.

A computer software product, the product comprising a non-transitory computer-readable medium, in which program instructions are stored, which instructions, when read by a computer, cause the computer:

• to receive, from a set of classifiers, respective predictions for a set of samples;
• to identify a subset of the samples having disagreements in their predictions;
• to receive labels for the samples in the subset;
• and
• to use the identified subset of the samples and their respective labels to compute a classification model comprising the set of classifiers.