# US 2018/0324196 A1 · System and Article of Manufacture to Analyze Twitter Data to Discover Suspicious Users and Malicious Content

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0324196 A1
- Document ID: US20180324196A1
- Status: Published application
- Kind: A1
- Application number: 15/585,397
- Filed: 2017-05-03
- Published: 2018-11-08
- Earliest claimed: 2017-05-03
- Assignees: The United States of America as represented by the Secretary of the Air Force
- Applicants: Goverment of the United States AS Represented By Te Secretary of the Air Force
- Inventors: PRAVEEN RAO; CHARLES KAMHOUA; LAURENT NJILLA; KEVIN KWIAT
- Web page: https://patentgrep.com/patent/US20180324196A1

## CPC

- H04L 63/1425: {Traffic logging, e.g. anomaly detection}
- G06N 5/04: Inference or reasoning models
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06N 20/00: Machine learning
- H04L 67/02: based on web technology, e.g. hypertext transfer protocol [HTTP]
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

The invention comprises a system and article of manufacture to discover potential cyber threats on Twitter. The invention provides a unified framework for modeling and reasoning about the veracity of tweets to discover suspicious users and malicious content. The invention builds on the concept of Markov logic networks (MLNs) for knowledge representation and reasoning under uncertainty.

## Claims

### Claim 1

An article of manufacture comprising a non-transitory storage medium having a plurality of programming instructions stored therein, the programming instructions being configured to program an apparatus to implement on the apparatus one or more subsystems or services to analyze Twitter data to discover suspicious users and malicious content by
collecting tweets from Twitter;
flagging malicious and benign URLs and domains;
generating ground predicates based on said tweets, said flagged URLs and domains, and a knowledge base of predicates and formulas;
generating a subset of said ground predicates based on an input set of queries;
learning weights of said formulas based on said subset of ground predicates, said tweets, and said knowledgebase;
updating said knowledgebase with first order predicates and formulas that capture the content of said tweets and the behavior of Twitter users;
performing probabilistic inference on said queries based on said updated knowledgebase;
combining results; and
outputting suspicious Twitter users and malicious Twitter content.

### Claim 2 (depends on 1)

The article of manufacture of claim 1 where said malicious and benign URLs and domains are collected from external data sources.

### Claim 3 (depends on 1)

The article of manufacture of claim 1 where said apparatus having one or more subsystems or services comprises at least one server.

### Claim 4 (depends on 1)

The article of manufacture of claim 1 where said predicates further comprise predicates with non-temporal attributes and predicates with temporal attributes.

### Claim 5 (depends on 4)

The article of manufacture of claim 4 where a said non-temporal predicate performs a function selected from the group consisting of functions that:
state whether a user posted a particular tweet;
indicate whether a tweet contains a particular URL;
state whether a particular user is mentioned in a tweet;
state whether a user retweeted a particular tweet;
state whether a tweet contains a particular hashtag;
state whether a user has been verified or not;
state whether a URL is malicious or not;
state whether a particular user ID has a particular friend user ID;
state whether a particular hashtag is trending;
state whether a particular user is suspicious or not;
state whether a particular user ID is followed by a particular friend user ID;
indicate whether a tweet is sensitive or not;
indicate whether a user has a particular number of friends;
indicate whether a user has a particular number of followers;
indicate whether a user has posted a particular number of tweets;
indicate whether a tweet has been retweeted a particular number of times; and
indicate whether a user has liked a particular number of tweets, or any combination thereof.

### Claim 6 (depends on 4)

The article of manufacture of claim 4 where a said temporal predicate performs a function selected from the group consisting of functions that:
indicate the approximate time a user posted a particular tweet;
indicate the approximate time a hashtag is trending;
indicate the approximate time a user has a particular number of followers;
indicate the approximate time a user has a particular number of friends;
indicate the approximate time a user has liked a particular number of tweets;
indicate the approximate time a user has tweeted a particular number of tweets; and
indicate the approximate time a tweet has been retweeted a particular number of times,
or any combination thereof.

### Claim 7 (depends on 1)

The article of manufacture of claim 1 where a said formula performs a function selected from the group consisting of functions that:
state that if a first user mentions a second user in his/her tweet, then said second user is deemed a friend of said first user;
state that if a first user retweets a tweet of a second user, then a friend relationship between said first user and said second user is implied;
state that if a user posted a hashtag and is an attacker/suspicious user, then said hashtag is trending;
state that if a first user is followed by a verified user, then said first user is deemed a verified/trustworthy user;
state that if a user is verified, then said user is not an attacker;
state that a friend of a verified user is not an attacker;
state that a user who posted a tweet containing a malicious link is an attacker;
state that a friend of an attacker is also an attacker;
state that if a first user, who is not an attacker, mentions a second user in said first user's tweet, then said second user is not an attacker; and
state that if a user's tweet is known to be possibly sensitive, then said user is an attacker,
or any combination thereof.

### Claim 8 (depends on 1)

The article of manufacture of claim 1 where a said formula performs a function selected from the group consisting of functions that:
state that a URL containing a certain prefix is not malicious;
state that a URL contained in a possibly sensitive tweet is malicious;
state that a URL in a tweet posted by an attacker is malicious;
state that a tweet containing a malicious URL is possibly sensitive; and
state that a tweet of an attacker is possibly sensitive,
or any combination thereof.

### Claim 9 (depends on 1)

The article of manufacture of claim 1 where a said formula performs a function selected from the group consisting of functions that:
state that if a non-verified user has a very large number of users he is following compared to the number of users following him, then said non-verified user is an attacker;
state that if a non-verified user is not deemed active on Twitter based on the number of posts but has a large number of friends, then said non-verified user is an attacker;
state that if a non-verified user is not deemed active on Twitter based on the number of posts but has a large number of followers, then said non-verified user is an attacker;
state that if a non-verified user is not deemed active on Twitter based on the number of posts but has liked a large number of tweets, then said non-verified user is an attacker;
state that if the friends count of a user increases substantially during a specified interval of time, then said user is a suspicious user attempting to increase his social influence;
state that if a hashtag is trending at a point in time, and an attacker posts a tweet containing said hashtag at a later time, and if said tweet contains a URL, then said URL deemed to be malicious; and
state that if a hashtag is trending at a point in time, and a first user posts a tweet containing said hashtag at a later time wherein said first user mentions a second user who said user is not following or is not friends with, then said first user is an attacker,
or any combination thereof.

### Claim 10 (depends on 4)

The article of manufacture of claim 4 where generating ground predicates further comprises:
for a said tweet, outputting said non-temporal ground predicates based on closed world assumptions and open world assumptions using said knowledgebase and external data sources;
for a said tweet, outputting said temporal ground predicates based on closed world assumptions and open world assumptions using said knowledgebase and external data sources; and
analyzing a next said tweet.

### Claim 11 (depends on 1)

The article of manufacture of claim 1 where generating ground predicates further comprises:
outputting a ground predicate based on the tweet I.D. and the user I.D. associated with said tweet;
outputting a ground predicate that indicates said user is verified when said user is verified;
outputting a ground predicate that indicates said user is not verified when said user in not verified; and
outputting for each URL contained in said tweet, a statement that said ground predicate contains links to said URL.

### Claim 12 (depends on 1)

The article of manufacture of claim 1 where generating ground predicates further comprises:
outputting for each particular hashtag contained in said tweet, a ground predicate that indicates said ground predicate contains said particular hashtag;
outputting for each particular user mentioned in said tweet, a ground predicate that indicates said ground predicate mentions said particular user; and
outputting for each particular user who retweeted said tweet, a ground predicate that indicates said particular user retweeted said tweet.

### Claim 13 (depends on 1)

The article of manufacture of claim 1 where generating ground predicates further comprises:
outputting for each particular URL contained in said tweet, a ground predicate that indicates said URL is malicious when said URL is known to be malicious;
outputting for each particular URL contained in said tweet, a ground predicate that indicates said URL is not malicious when said URL is known to be benign;
outputting for each known friend of a particular user, a ground predicate that indicates said friendship;
outputting for each known follower of a particular user, a ground predicate that indicates said following;
outputting for each particular hashtag contained in said tweet, a ground predicate that indicates said particular hashtag is trending when said particular hashtag is trending;
outputting for each particular user known to be an attacker, a ground predicate that indicates said particular user is an attacker; and
outputting for each tweet that is possibly sensitive, a ground predicate that indicates said tweet is possibly sensitive.

### Claim 14 (depends on 1)

The article of manufacture of claim 1 where generating ground predicates further comprises outputting a ground predicate that performs a function selected from the group consisting of functions that denote:
the friends count of said user;
the followers count of said user;
the statuses count of said user;
the favorites count of said user; and
the retweet count of a tweet,
or any combination thereof.

### Claim 15 (depends on 4)

The article of manufacture of claim 4, where generating ground predicates with temporal attributes further comprises outputting a ground predicate that performs a function selected from the group consisting of functions that denote:
a timestamp of a tweet;
an identification of a hashtag that is trending at approximately the time of said timestamp;
a count of friends at approximately the time of said timestamp;
a count of followers at approximately the time of said timestamp;
a count of statuses at approximately the time of said timestamp;
a count of favorites at approximately the time of said timestamp; and
a count of retweets at approximately the time of said timestamp,
or any combination thereof.

### Claim 16 (depends on 4)

The article of manufacture of claim 4 where combining results further comprises
ranking those ground predicates generated by maximum a posteriori inference according to probabilities generated by marginal inference;
outputting those said ground predicates higher than a first predetermined probability;
ranking those ground predicates not having been generated by maximum a posteriori inference according to probabilities generated by marginal inference; and
outputting those said ground predicates higher than a second predetermined probability.

### Claim 17 (depends on 1)

The article of manufacture of claim 1 where results are combined through a Markov Logic Network process on said learned weights and said probabilistic inference.

### Claim 18

A system for analyzing Twitter data to discover suspicious users and malicious content, comprising:
at least one computer server executing a plurality of programming instructions stored therein;
at least one source of externally stored data;
a communications channel for receiving tweets from Twitter; and
a user interface for querying said system; and
said programming instructions configured to program said computer server to perform tasks, said tasks further comprising communicating with:
said at least one source of external data,
said communications channel for receiving tweets from Twitter, and
said user interface
so as to identify and output suspicious Twitter users and malicious Twitter content.

### Claim 19 (depends on 18)

The system of claim 18 where said tasks further comprise:
generating ground predicates based on:
received tweets,
Twitter users' information,
malicious and benign URLs, and
a knowledge base of predicates and formulas;
updating said knowledge base with first order formulas to capture the nature of the content in said tweets and Twitter users' behavior;
defining an input set of queries to learn weights of the said formulas on said ground predicates;
performing probabilistic inference on said queries based on said updated knowledge base;
combining results; and
outputting suspicious Twitter users and malicious Twitter content.

### Claim 20 (depends on 19)

The system of claim 19 where said predicates further comprise predicates with non-temporal attributes and predicates with temporal attributes.

### Claim 21 (depends on 19)

The system of claim 19 where a said non-temporal predicate performs a function selected from the group consisting of functions that:
state whether a user posted a particular tweet;
indicate whether a tweet contains a particular URL;
state whether a particular user is mentioned in a tweet;
state whether a user retweeted a particular tweet;
state whether a tweet contains a particular hashtag;
state whether a user has been verified;
state whether a URL is malicious or not;
state whether a particular user ID has a particular friend user ID;
state whether a particular hashtag is trending;
state whether a particular user is suspicious or not;
state whether a particular user ID is followed by a particular friend user ID;
indicate whether a tweet is sensitive or not;
indicate whether a user has a particular number of friends;
indicate whether a user has a particular number of followers;
indicate whether a user has posted a particular number of tweets;
indicate whether a tweet has been retweeted a particular number of times; and
indicate whether a user has liked a particular number of tweets,
or any combination thereof.

### Claim 22 (depends on 19)

The system of claim 19 where a said temporal predicate performs a function selected from the group consisting of functions that:
indicate the approximate time a user posted a particular tweet;
indicate the approximate time a hashtag is trending;
indicate the approximate time a user has a particular number of followers;
indicate the approximate time a user has a particular number of friends;
indicate the approximate time a user has liked a particular number of tweets;
indicate the approximate time a user has tweeted a particular number of tweets; and
indicate the approximate time a tweet has been retweeted a particular number of times,
or any combination thereof.

### Claim 23 (depends on 19)

The system of claim 19 where a said formula performs a function selected from the group consisting of functions that:
state that if a first user mentions a second user in his/her tweet, then said second user is deemed a friend of said first user;
state that if a first user retweets a tweet of a second user, then a friend relationship between said first user and said second user is implied;
state that if a user posted a hashtag and is an attacker/suspicious user, then said hashtag is trending;
state that if a first user is followed by a verified user, then said first user is deemed a verified/trustworthy user;
state that if a user is verified, then said user is not an attacker;
state that a friend of a verified user is not an attacker;
state that a user who posted a tweet containing a malicious link is an attacker;
state that a friend of an attacker is also an attacker;
state that if a first user, who is not an attacker, mentions a second user in said first user's tweet, then said second user is not an attacker; and
state that if a user's tweet is known to be possibly sensitive, then said user is an attacker,
or any combination thereof.

### Claim 24 (depends on 19)

The system of claim 19 where a said formula performs a function selected from the group consisting of functions that:
state that a URL containing a certain prefix is not malicious;
state that a URL contained in a possibly sensitive tweet is malicious;
state that a URL in a tweet posted by an attacker is malicious;
state that a tweet containing a malicious URL is possibly sensitive; and
state that a tweet of an attacker is possibly sensitive,
or any combination thereof.

### Claim 25 (depends on 19)

The system of claim 19 where a said formula performs a function selected from the group consisting of functions that:
state that if a non-verified user has a very large number of users he is following compared to the number of users following him, then said non-verified user is an attacker;
state that if a non-verified user is not deemed active on Twitter based on the number of posts but has a large number of friends, then said non-verified user is an attacker;
state that if a non-verified user is not deemed active on Twitter based on the number of posts but has a large number of followers, then said non-verified user is an attacker;
state that if a non-verified user is not deemed active on Twitter based on the number of posts but has liked a large number of tweets, then said non-verified user is an attacker;
state that if the friends count of a user increases substantially during a specified interval of time, then said user is a suspicious user attempting to increase his social influence;
state that if a hashtag is trending at a point in time, and an attacker posts a tweet containing said hashtag at a later time, and if said tweet contains a URL, then said URL deemed to be malicious; and
state that if a hashtag is trending at a point in time, and a first user posts a tweet containing said hashtag at a later time wherein said first user mentions a second user who said user is not following or is not friends with, then said first user is an attacker,
or any combination thereof.

### Claim 26 (depends on 19)

The system of claim 19 where generating ground predicates further comprises:
for a said tweet, outputting said non-temporal ground predicates based on closed world assumptions and open world assumptions using said knowledgebase and external data sources;
for a said tweet, outputting said temporal ground predicates based on closed world assumptions and open world assumptions using said knowledgebase and external data sources; and
analyzing a next said tweet.

### Claim 27 (depends on 19)

The system of claim 19 where generating ground predicates further comprises:
outputting a ground predicate based on the tweet I.D. and the user I.D. associated with said tweet;
outputting a ground predicate that indicates said user is verified when said user is verified;
outputting a ground predicate that indicates said user is not verified when said user is not verified; and
outputting for each URL contained in said tweet, a statement that said ground predicate contains links to said URL.

### Claim 28 (depends on 19)

The system of claim 19 where generating ground predicates further comprises:
outputting for each particular hashtag contained in said tweet, a ground predicate that indicates said ground predicate contains said particular hashtag;
outputting for each particular user mentioned in said tweet, a ground predicate that indicates said ground predicate mentions said particular user; and
outputting for each particular user who retweeted said tweet, a ground predicate that indicates said particular user retweeted said tweet.

### Claim 29 (depends on 19)

The system of claim 19 where generating ground predicates further comprises:
outputting for each particular URL contained in said tweet, a ground predicate that indicates said URL is malicious when said URL is known to be malicious;
outputting for each particular URL contained in said tweet, a ground predicate that indicates said URL is not malicious when said URL is known to be benign;
outputting for each known friend of a particular user, a ground predicate that indicates said friendship;
outputting for each known follower of a particular user, a ground predicate that indicates said following;
outputting for each particular hashtag contained in said tweet, a ground predicate that indicates said particular hashtag is trending when said particular hashtag is trending;
outputting for each particular user known to be an attacker, a ground predicate that indicates said particular user is an attacker; and
outputting for each tweet that is possibly sensitive, a ground predicate that indicates said tweet is possibly sensitive.

### Claim 30 (depends on 19)

The system of claim 19 where generating ground predicates further comprises outputting a ground predicate that denotes at least one of the following:
the friends count of said user;
the followers count of said user;
the statuses count of said user;
the favorites count of said user; and
the retweet count of a tweet,
or any combination thereof.

### Claim 31 (depends on 19)

The system of claim 19 where generating ground predicates with temporal attributes further comprises outputting a ground predicate that performs a function selected from the group consisting of functions that denote:
a timestamp of a tweet;
an identification of a hashtag that is trending at approximately the time of said timestamp;
a count of friends at approximately the time of said timestamp;
a count of followers at approximately the time of said timestamp;
a count of statuses at approximately the time of said timestamp;
a count of favorites at approximately the time of said timestamp; and
a count of retweets at approximately the time of said timestamp,
or any combination thereof.

### Claim 32 (depends on 19)

The system of claim 19 where combining results further comprises
ranking those ground predicates generated by maximum a posteriori inference according to probabilities generated by marginal inference;
outputting those said ground predicates higher than a first predetermined probability;
ranking those ground predicates not having been generated by maximum a posteriori inference according to probabilities generated by marginal inference; and
outputting those said ground predicates higher than a second predetermined probability.
