US 8,788,407 B1Granted

Malware data clustering

AssigneePalantir Technologies Inc.

InventorsHarkirat Singh, Geoff Stowe, Brendan Weickert, Matthew Sprague, Michael Kross, Adam Borochoff, Parvathy Menon, Michael Harris

Life of this patent

Estimating the term…

patentgrep

Loading patent…

Abstract

In various embodiments, systems, methods, and techniques are disclosed for generating a collection of clusters of related data from a seed. Seeds may be generated based on seed generation strategies or rules. Clusters may be generated by, for example, retrieving a seed, adding the seed to a first cluster, retrieving a clustering strategy or rules, and adding related data and/or data entities to the cluster based on the clustering strategy. Various cluster scores may be generated based on attributes of data in a given cluster. Further, cluster metascores may be generated based on various cluster scores associated with a cluster. Clusters may be ranked based on cluster metascores. Various embodiments may enable an analyst to discover various insights related to data clusters, and may be applicable to various tasks including, for example, tax fraud detection, beaconing malware detection, malware user-agent detection, and/or activity trend detection, among various others.

Claims19

Dependent claims

A computer system comprising:

one or more computer readable storage devices configured to store:
one or more software modules including computer executable instructions;
a clustering strategy;
and
a plurality of beaconing malware-related data items and properties associated with respective malware-related data items, each of the properties including associated property values, the beaconing malware-related data items including at least one of:
data items associated with captured communications between an internal network and an external network, users of particular computerized devices, internal Internet Protocol addresses, external Internet Protocol addresses, external domains, internal computerized devices, external computerized devices, data feed items, or host-based events;
and
one or more hardware computer processors in communication with the one or more computer readable storage devices and configured to execute the one or more software modules in order to cause the one or more hardware computer processors to:
access, from the one or more computer readable storage devices, the plurality of beaconing malware-related data items;
generate, based on the accessed beaconing malware-related data items, a plurality of connection pairs, each of the connection pairs indicating communications between a particular internal source within the internal network and a particular external destination that is not within the internal network;
identify a plurality of connection pairs having a common internal source and a common external destination;
generate a time series of connection pairs based on the identified plurality of connection pairs;
filter out noise from the at least one time series to generate a filtered at least one time series;
compute a variance in the filtered at least one time series;
and
based on a determination that the variance satisfies a particular threshold:
designate a connection pair associated with the filtered at least one time series as a seed, the designated connection pair including the common internal source and the common external source;
generate a data item cluster based on the designated seed, wherein generating the data item cluster comprises:
adding the designated seed to the data item cluster;
accessing, from the one or more computer readable storage devices, the clustering strategy;
and
adding to the data item cluster, based on the clustering strategy, one or more beaconing malware-related data items determined to be associated with the designated seed;
and
score the generated data item cluster.

21 more elements

A computer-implemented method comprising:

storing on one or more computer readable storage devices:
a clustering strategy;
and
a plurality of beaconing malware-related data items and properties associated with respective malware-related data items, each of the properties including associated property values, the beaconing malware-related data items including at least one data item associated with captured communications between an internal network and an external network;
accessing, by one or more hardware computer devices configured with specific computer executable instructions, and from the one or more computer readable storage devices, the plurality of beaconing malware-related data items;
generating, by the one or more hardware computer devices and based on the accessed beaconing malware-related data items, a plurality of connection pairs, each of the connection pairs indicating communications between a particular internal source within the internal network and a particular external destination that is not within the internal network;
identifying, by the one or more hardware computer devices, a plurality of connection pairs having a common internal source and a common external destination;
generating, by the one or more hardware computer devices, a time series of connection pairs based on the identified plurality of connection pairs;
filtering out, by the one or more hardware computer devices, noise from the at least one time series to generate a filtered at least one time series;
computing, by the one or more hardware computer devices, a variance in the filtered at least one time series;
and
based on a determination that the variance satisfies a particular threshold:
designating, by the one or more hardware computer devices, a connection pair associated with the filtered at least one time series as a seed, the designated connection pair including the common internal source and the common external source;
generating, by the one or more hardware computer devices, a data item cluster based on the designated seed, wherein generating the data item cluster comprises:
adding the designated seed to the data item cluster;
accessing, from the one or more computer readable storage devices, the clustering strategy;
and
adding to the data item cluster, based on the clustering strategy, one or more beaconing malware-related data items determined to be associated with the designated seed;
and
scoring, by the one or more hardware computer devices, the generated data item cluster.

17 more elements

A non-transitory computer-readable storage medium storing computer-executable instructions that, when executed by a computer system, configure the computer system to perform operations comprising:

storing on one or more computer readable storage devices:
a clustering strategy;
and
a plurality of beaconing malware-related data items and properties associated with respective malware-related data items, each of the properties including associated property values, the beaconing malware-related data items including at least one data item associated with captured communications between an internal network and an external network;
accessing, from the one or more computer readable storage devices, the plurality of beaconing malware-related data items;
generating, based on the accessed beaconing malware-related data items, a plurality of connection pairs, each of the connection pairs indicating communications between a particular internal source within the internal network and a particular external destination that is not within the internal network;
identifying a plurality of connection pairs having a common internal source and a common external destination;
generating a time series of connection pairs based on the identified plurality of connection pairs;
filtering out noise from the at least one time series to generate a filtered at least one time series;
computing a variance in the filtered at least one time series;
and
based on a determination that the variance satisfies a particular threshold:
designating a connection pair associated with the filtered at least one time series as a seed, the designated connection pair including the common internal source and the common external source;
generating a data item cluster based on the designated seed, wherein generating the data item cluster comprises:
adding the designated seed to the data item cluster;
accessing, from the one or more computer readable storage devices, the clustering strategy;
and
adding to the data item cluster, based on the clustering strategy, one or more beaconing malware-related data items determined to be associated with the designated seed;
and
scoring the generated data item cluster.

17 more elements

Dependencies are inferred from the claim text.

Facts

Application: 14/139,603Filed 23 Dec 2013 · continuation in part
Priority: 15 Mar 2013
Granted: 22 Jul 2014
Term ends: Not estimatedThe source records don’t support an estimate.

Classification26 codes

H04L 63/145
Countermeasures against malicious traffic: the attack involving the propagation of malware through the network, e.g. viruses, trojans or worms
G06F 16/23
Of structured data, e.g. relational data: updating
G06F 16/244
Query languages: grouping and aggregation
G06F 16/24578
With adaptation to user needs: using ranking

Family

Application61/800,887Us provisional application · filed 15 Mar 2013
Application13/968,265Parent of this continuation-in-part · filed 15 Aug 2013
Application13/968,213Parent of this continuation-in-part · filed 15 Aug 2013
US 8,712,906 B1Related patent
US 8,788,405 B1Related patent
US 8,818,892 B1Related patent
US 8,788,407 B1This patent

From the US filing history. Foreign family members aren’t listed.