Large scale malware sample identification

At least one non-transitory computer-readable medium comprising one or more instructions that when executed by a processor, cause the processor to:

• receive a feature vector of an unknown malware, wherein the feature vector includes Jaccard-compatible features, a first non-Jaccard sub-distance feature, and a second non-Jaccard sub-distance feature;
• compute a MinHash of the unknown malware;
• identify a first subset of known malware, wherein identifying the first subset includes querying a MinHash Locality Sensitive Hashing (LSH) Forest of known malware with the MinHash of the unknown malware;
• receive a feature vector for each individual known malware in the first subset, wherein the feature vector includes a first non-Jaccard sub-distance feature and a second non-Jaccard sub-distance feature;
• calculate a first sub-distance and a second sub-distance between the unknown malware and each individual known malware in the first subset, wherein the first sub-distance is based on the first non-Jaccard sub-distance features and the second sub-distance is based on the second non-Jaccard sub-distance features;
• calculate a Custom Distance Metric (CDM) for each individual known malware in the first subset by combining the first and the second sub-distances;
• identify, using the calculated CDM for each individual known malware in the first subset, a second subset of known malware that are most similar to the unknown malware;
• and
• classify the unknown malware based on the second subset of known malware.

A method, comprising:

• receiving a MinHash for an unknown malware sample;
• querying a MinHash Locality Sensitive Hashing (LSH) Forest using the MinHash for the unknown malware sample to identify a first set of similar samples, wherein the MinHash LSH Forest is based on a plurality of known malware samples, and wherein each known malware sample in the plurality of known malware samples includes a unique hashing identifier and a MinHash;
• receiving a first non-Jaccard distance-compatible feature and a second non-Jaccard distance-compatible feature for the unknown malware sample;
• receiving a first non-Jaccard distance-compatible feature and a second non-Jaccard distance-compatible feature for each known malware sample in the first set of similar samples;
• calculating a total distance between the unknown malware sample and each known malware sample in the first set of similar samples, wherein the total distance is a combination of a first distance and a second distance, and wherein the first distance is calculated using the first non-Jaccard distance-compatible features and the second distance is calculated using the second non-Jaccard distance-compatible features;
• identifying, based on the total distance for each known malware sample, a second set of similar samples;
• and
• classifying the unknown malware sample based on the second set of similar samples.