System and method for the automated detection and prediction of online threats

A method for alerting a client system of a potential malware threat, comprising:

• receiving a first set of network traffic data comprising past internet threat data;
• building a threat detection model by:
• extracting a first set of network related features from the first set of network traffic data, wherein the first set of network related features comprises one or more queueing-based features that include an infection arrival rate and an infection service rate;
• clustering the first set of network related features based on their network size;
• training a respective threat classifier for each cluster of the first set of network related features by (i) constructing a first training set that comprises a first positive set and a first negative set, wherein the first positive set comprises first malicious samples and the first negative set comprises first benign samples, and (ii) detecting patterns in the first malicious samples based at least in part on the infection arrival rate being greater than or less than a first threshold rate and the infection service rate being greater than or less than a second threshold rate;
• evaluating accuracy of each of the respective trained threat classifiers of the threat detection model on a test data set;
• building a threat prediction model by:
• extracting a first set of registry related features from the first set of network traffic data, wherein the first set of registry related features comprises at least one of a contact entropy-based feature and a passive domain name system (pDNS)-based feature;
• clustering the first set of registry related features based on their network size;
• training a respective threat classifier for each cluster of the first set of registry related features by (i) constructing a second training set that comprises a second positive set and a second negative set, wherein the second positive set comprises second malicious samples and the second negative set comprises second benign samples, and (ii) detecting patterns in the malicious samples based at least in part on one or more of an entropy-based feature and a pDNS-based feature;
• evaluating accuracy of each of the respective trained threat classifiers of the threat prediction model on the test data set;
• receiving a second set of network traffic data comprising contemporaneous internet data including IP address data;
• extracting a second set of network related features and a second set of registry related features from the second set of network traffic data;
• clustering the second set of network related features and the second set of registry related features based on their Classless Inter-Domain Routing (CIDR) size;
• comparing each cluster of the second set of network related features to a respective threat classifier of the respective trained threat classifiers of the threat detection model to determine a malice score for each IP address in the IP address data included in the second set of network traffic data;
• comparing each cluster of the second set of registry related features to a respective threat classifier of the respective trained threat classifiers of the threat prediction model to generate a predictive threat list of IP addresses in the second set of network traffic data that are likely to be used in future attacks;
• and
• alerting the client system of the predictive threat list and the determined malice score for each IP address in the IP address data.