Apparatus and methods for remote classification of unknown malware

A method performed by executing computer-readable code on an apparatus which includes a data storage system to store the computer-readable code and data, a network interface communicatively connected to a network to receive and send data via the network, and a processor to access the data storage system and to execute said computer-readable code, the method comprising:

• said apparatus receiving a locality-sensitive hash (LSH) value and a checksum of a file from a host computer via the network interface;
• said apparatus determining whether the LSH value of the file is similar to a LSH value in an entry in an LSH data structure;
• said apparatus indicating that the file is a variant of known malware of the entry if the LSH value of the file is similar to the LSH value in the entry while the checksum of the file is different from the checksum in the entry;
• said apparatus determining that the file is a variant of an unknown malware when the LSH value of the file is similar to a group of LSH values that were previously received but not determined to be malware such that the LSH values in the group are not in any entry in the LSH data structure, wherein the group of LSH values has an adjustable predetermined number of matching bits, while the checksum of the file is different from checksums of the group;
• and
• said apparatus indicating that files of the group of LSH values are variants of an unknown malware which is not of any entry in the LSH data structure.

A system for determining whether a file includes malware, the system comprising:

• a host computer which includes
• a data storage system to store computer-readable code and data,
• a network interface communicatively connected to a network and configured to receive and send data via the network,
• a processor to access the data storage system and to execute said computer-readable code,
• computer-readable code for execution by the processor so as to select a file to be checked for presence of malware,
• computer-readable code for execution by the processor so as to calculate a locality-sensitive hash (LSH) value and a checksum of the file,
• computer-readable code for execution by the processor so as to send the LSH value of the file to a remote malware classifier, and
• computer-readable code for execution by the processor so as to receive a result from the remote malware classifier which indicates whether the file includes malware;
• and
• a server computer which includes the remote malware classifier, wherein the remote malware classifier includes computer-readable code for determining whether the LSH value of the file is similar to an LSH value in an entry in an LSH data structure, computer-readable code for indicating that the file is a variant of known malware of the entry if the LSH value of the file is similar to the LSH value in the entry while the checksum of the file is different from the checksum in the entry, computer-readable code for determining that the file is a variant of an unknown malware when the LSH value associated with the file is similar to a group of LSH values that were previously received by the server computer but not determined to be malware such that the LSH values in the group are not in any entry in the LSH data structure, wherein the group of LSH values has an adjustable predetermined number of matching bits, while the checksum of the file is different from checksums of the group, and computer-readable code for indicating that the file is a variant of an unknown malware which is not of any entry in the LSH data structure.