Scalable training of random forests for high precise malware detection

An apparatus comprising:

• one or more network interfaces to communicate with a network;
• a processor coupled to the network interfaces and configured to execute one or more processes;
• and
• a memory configured to store a process executable by the processor, the process when executed configured to:
• train a machine learning-based malware classifier using a first randomly selected subset of samples from a training dataset, wherein the classifier comprises a random decision forest;
• identify, using at least a portion of the training dataset as input to the malware classifier, a set of misclassified samples from the training dataset that the malware classifier misclassifies;
• retrain the malware classifier using a second randomly selected subset of samples from the training dataset and the identified set of misclassified samples;
• adjust prediction labels of individual leaves of the random decision forest of the retrained malware classifier based in part on decision changes in the forest that result from assessing the entire training dataset with the classifier;
• and
• send the malware classifier with the adjusted prediction labels for deployment into a network.

A tangible, non-transitory, computer-readable medium storing program instructions that cause a device to execute a process comprising:

• training, by the device, a machine learning-based malware classifier using a first randomly selected subset of samples from a training dataset, wherein the classifier comprises a random decision forest;
• identifying, by the device and using at least a portion of the training dataset as input to the malware classifier, a set of misclassified samples from the training dataset that the malware classifier misclassifies;
• retraining, by the device, the malware classifier using a second randomly selected subset of samples from the training dataset and the identified set of misclassified samples;
• adjusting, by the device, prediction labels of individual leaves of the random decision forest of the retrained malware classifier based in part on decision changes in the forest that result from assessing the entire training dataset with the classifier;
• and
• sending, by the device, the malware classifier with the adjusted prediction labels for deployment into a network.