Distributed systems and methods for automatically detecting unknown bots and botnets

A computer implemented method for detecting callbacks from malicious code, comprising:

• A) performing, by a local analyzer,
• (i) signature matching of a generated header signature, based on information from a header obtained from a captured packet of a plurality of captured packets, against a plurality of packet header signatures corresponding to verified callbacks stored in a local cache, and
• (ii) first stage filtering of the header to detect whether the header corresponds to a suspect header upon detecting that the header includes header anomalies;
• and
• B) performing, by a central analyzer,
• (i) signature matching of the header signature for the suspect header received from the local analyzer against a plurality of header signatures corresponding to verified callbacks stored in a central cache,
• (ii) second stage filtering of the suspect header to detect suspicious characteristics associated with the suspect header, the suspect header detected to have suspicious characteristics being a suspicious suspect header,
• (iii) generating a probability score based on the suspicious characteristics for each of the suspicious suspect header,
• (iv) comparing the probability score with a threshold to verify whether the suspicious suspect header corresponds to a callback, and
• (v) storing the header signature of the suspicious suspect header verified as corresponding to a callback in the central cache, and sending a message to the local analyzer to update the local cache.

A computer implemented method performed by a local analyzer for detecting callbacks from malicious code, the local analyzer comprises one or more processors and software stored in a memory accessible to the one or more processors, and is configured for communications over a network with a central analyzer, the method comprising:

• A) generating a header signature based on information from a header obtained from a captured packet of a plurality of captured packets;
• B) signature matching of the header signature against a plurality of packet header signatures corresponding to verified callbacks stored in a local cache of the local analyzer;
• C) responsive to the header signature corresponding to none of the plurality of packet header signatures, performing a first stage filtering of the captured packet headers of the plurality of captured packets to detect whether the captured packet header includes with header anomalies, the captured packet header identified as having header anomalies being a suspect header;
• D) sending the suspect header and the header signature to the central analyzer, and receiving from the central analyzer a message providing information for the suspect header verified through a second stage filtering by the central analyzer as corresponding to a callback;
• and
• E) updating the local cache with the information received from the central analyzer, thereby storing in the local cache an updated listing for the plurality of packet header signatures corresponding to verified callbacks.

A computer implemented method performed by a central analyzer for detecting callbacks from malicious code, the central analyzer configured for communication over a network with a plurality of local analyzers, the method comprising:

• A) receiving information identifying anomalies within a plurality of suspect headers from a local analyzer;
• B) comparing a header signature for each of the plurality of suspect headers received from the local analyzer against a plurality of header signatures corresponding to verified callbacks stored in a global cache associated with central analyzer;
• C) conducting an analysis of each of the plurality of suspect headers to detect whether any of the plurality of suspect headers has at least one suspicious attribute, the suspect headers detected to have suspicious attributes being suspicious suspect headers;
• D) verifying whether any of suspicious suspect headers correspond to callbacks, wherein the verifying comprises generating a probability score based on the anomalies and the at least one suspicious attribute for each of the suspicious suspect headers, and comparing the probability score with a threshold in determining whether any of the suspicious suspect headers should be classified as a callback;
• E) storing the header signatures of the suspicious suspect headers verified as corresponding to callbacks in the global cache;
• and
• F) sending a message to the local analyzer containing information with respect to the suspicious suspect headers verified as corresponding to callbacks.

A local analyzer for detecting callbacks from malicious code and configured for communication over a network with a central analyzer, the local analyzer comprising:

• one or more processors;
• and
• a persistent storage device coupled to the one or more processors, the persistent storage device comprises
• a signature generating logic that, when executed by the one or more processors, is configured to generate a header signature based on information from a header obtained from a captured packet of a plurality of captured packets,
• a signature matching logic that, when executed by the one or more processors, is configured to match the header signature against a plurality of packet header signatures corresponding to verified callbacks stored in a local cache of the local analyzer,
• a first stage filter that, when executed by the one or more processors and in response to the header signature failing to match any of the plurality of packet header signatures, is configured to detect whether the captured packet header includes one or more header anomalies, the captured packet header identified as having one or more header anomalies being a suspect header;
• and
• a network interface configured to send suspect header and the header signature to the central analyzer, and to receive from the central analyzer a message providing information for each suspect header verified through a second stage filter by the central analyzer as corresponding to a callback,
• wherein the persistent storage device further comprises caching control logic that, when executed by the one or more processors, is configured to update the local cache with the information received from the central analyzer, thereby storing in the local cache an updated listing for packet header signatures corresponding to verified callbacks.

A central analyzer for detecting callbacks from malicious code and for communication over a network with a plurality of local analyzers, the central analyzer comprising:

• a network interface operable to receive information identifying a number of anomalies from a local analyzer;
• one or more processors;
• and
• a persistent storage device coupled to the one or more processors, the persistent storage device comprises
• a signature matching logic that, when executed by the one or more processors, is configured to match a header signature for each of a plurality of suspect headers received from the local analyzer against a plurality of header signatures corresponding to verified callbacks stored in a global cache associated with the central analyzer,
• a stage filter of the central analyzer that, when executed by the one or more processors, is configured to filter the plurality of suspect headers to detect suspicious attributes related thereto, the plurality of suspect headers detected to have suspicious attributes being suspicious suspect headers, and
• logic that, when executed by the one or more processors, is configured to generate a probability score based on the received anomalies and the suspicious attributes for a corresponding suspicious suspect header, to compare the probability score with a threshold in determining whether the suspicious suspect header should be classified as a callback, and to store the header signature of the corresponding suspicious suspect header classified as a callback in the global cache, wherein
• the network interface further operable to output a message containing information with respect to the suspicious suspect header verified as corresponding to a callback.

A non-transitory machine readable medium storing instructions, which when executed by a processor, causes the processor to perform operations for detecting callbacks from malicious code at a central analyzer, comprising:

• A) generating a header signature based on information from a header obtained from each of a plurality of captured packets;
• B) signature matching of the header signature against a plurality of packet header signatures corresponding to verified callbacks stored in a local cache of the local analyzer;
• C) first stage filtering of the captured packet headers to detect those with header anomalies, the packet headers identified as having header anomalies being suspect headers;
• D) sending suspect headers and the header signatures to the central analyzer;
• E) generating a probability score based on the header anomalies for each of the suspect headers;
• F) comparing the probability score with a threshold to verify whether the suspicious network header corresponds to a callback;
• G) receiving from the central analyzer a message providing information for each of the suspect headers verified by the central analyzer as corresponding to a callback;
• and
• H) updating the local cache with the information received from the central analyzer, thereby storing in the local cache an updated listing for packet header signatures corresponding to verified callbacks.