# US 2004/0177110 A1 · Feedback loop for spam prevention

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2004/0177110 A1
- Document ID: US20040177110A1
- Status: Published application
- Kind: A1
- Application number: 10/378,463
- Filed: 2003-03-03
- Published: 2004-09-09
- Earliest claimed: 2003-03-03
- Assignees: Microsoft Corporation
- Inventors: Robert L. Rounthwaite; Joshua T. Goodman; David E. Heckerman; John D. Mehr; Nathan D. Howell; Micah C. Rupersburg; Dean A. Slawson
- Web page: https://patentgrep.com/patent/US20040177110A1

## CPC

- H04L 51/212: using filtering or selective blocking
- G06Q 10/107: Computer-aided management of electronic mailing [e-mailing]
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

The subject invention provides for a feedback loop system and method that facilitate classifying items in connection with spam prevention in server and/or client-based architectures. The invention makes uses of a machine-learning approach as applied to spam filters, and in particular, randomly samples incoming email messages so that examples of both legitimate and junk/spam mail are obtained to generate sets of training data. Users which are identified as spam-fighters are asked to vote on whether a selection of their incoming email messages is individually either legitimate mail or junk mail. A database stores the properties for each mail and voting transaction such as user information, message properties and content summary, and polling results for each message to generate training data for machine learning systems. The machine learning systems facilitate creating improved spam filter(s) that are trained to recognize both legitimate mail and spam mail and to distinguish between them.

## Claims

### Claim 1

A system that facilitates classifying items in connection with spam prevention, comprising:
a component that receives a set of the items;
a component that identifies intended recipients of the items, and tags a subset of the items to be polled, the subset of items corresponding to a subset of recipients that are known spam fighting users; and
a feedback component that receives information relating to the spam fighter's classification of the polled items, and employs the information in connection with training a spam filter and populating a spam list.

### Claim 2 (depends on 1)

The system of claim 1, wherein the items comprise at least one of: electronic mail (email) and messages.

### Claim 3 (depends on 1)

The system of claim 1, wherein the component that receives a set of the items is any one of an email server, a message server, and client email software.

### Claim 4 (depends on 1)

The system of claim 1, wherein the subset of items to be polled comprises all of the items received.

### Claim 5 (depends on 1)

The system of claim 1, wherein the subset of recipients comprises all recipients.

### Claim 6 (depends on 1)

The system of claim 1, wherein the subset of recipients are randomly selected.

### Claim 7 (depends on 1)

The system of claim 1, wherein the subset of recipients comprises paying users of the system.

### Claim 8 (depends on 1)

The system of claim 1, wherein at least a subset of messages that would ordinarily be filtered are considered for polling.

### Claim 9 (depends on 1)

The system of claim 1, wherein the subset of items tagged for polling is limited to at least one of the following:
a number of the items selected per user;
a number of the items selected per user per time period; and
a probability of tagging an item corresponding to a known user.

### Claim 10 (depends on 1)

The system of claim 1, wherein the tagged items are each assigned a unique ID, the unique ID corresponding to any one of the tagged item and contents of the tagged item.

### Claim 11 (depends on 1)

The system of claim 1, further comprising a component that modifies an item tagged for polling to identify it as a polling item.

### Claim 12 (depends on 11)

The system of claim 11, wherein the modified item comprises at least one of the following:
a modified “from” address;
a modified subject line;
a polling icon; and
a polling color for identification as a polling item.

### Claim 13 (depends on 11)

The system of claim 11, wherein the modified item comprises the tagged item as an attachment.

### Claim 14 (depends on 11)

The system of claim 11, wherein the modified item comprises a summary of the tagged item, the summary comprising at least one of a subject, a date, text of the message, and a first few lines of the text.

### Claim 15 (depends on 11)

The system of claim 11, wherein the modified item comprises voting instructions and any one of at least two voting buttons and links which correspond to at least two respective classes of items facilitate classification of the item by the user.

### Claim 16 (depends on 15)

The system of claim 15, wherein the voting buttons correspond to respective links such that when any one of the voting buttons is selected by the user, information relating to the selected voting button, the respective user, and the item's unique ID assigned thereto is sent to a database for storage.

### Claim 17 (depends on 15)

The system of claim 15, wherein the at least two voting buttons comprise a first voting button, the first voting button indicating “legitimate mail” and a second voting button, the second voting button indicating “spam”.

### Claim 18 (depends on 15)

The system of claim 15, wherein the voting buttons are implemented by modifying text of the item.

### Claim 19 (depends on 15)

The system of claim 15, wherein the voting buttons are implemented by modifying a user interface of client email software.

### Claim 20 (depends on 1)

The system of claim 1, further comprising a central database that stores information and data relating to user properties, item content and properties associated with tagged items, user classification and voting statistical data, frequency analysis data of polling per user and of polling per user per time period, spam lists, legitimate mail lists, and black hole lists.

### Claim 21 (depends on 1)

The system of claim 1, wherein items tagged for polling that are marked as spam by an existing filter are delivered to the user's inbox and considered for polling.

### Claim 22 (depends on 1)

The system of claim 1, wherein items tagged for polling are scanned for viruses such that one of the following occurs:
detected viruses are stripped out and the items are polled; and
infected items are discarded.

### Claim 23 (depends on 1)

The system of claim 1 distributed across more than one spam-fighting company such that feedback from each company is sent to a central database operatively interfaced with each company, wherein some portion of the feedback is removed for privacy reasons.

### Claim 24 (depends on 23)

The system of claim 23, wherein the company feedback comprises one of the following:
only spam items, thereby excluding legitimate items; and
spam items and sender name, domain name and IP address of legitimate items.

### Claim 25 (depends on 1)

The system of claim 1, further comprising a user classification validation component that tests user reliability and trustworthiness.

### Claim 26 (depends on 25)

The system of claim 25, wherein the user classification validation component is at least one of a cross-validation technique and a known result test message technique.

### Claim 27 (depends on 25)

The system of claim 25, wherein the user classification validation component can be applied to one or more suspected users.

### Claim 28 (depends on 1)

The system of claim 1, wherein the feedback component receives information relating to user feedback, honeypot feedback, and optionally, user recipient feedback of received items.

### Claim 29 (depends on 1)

A server employing the system of claim 1.

### Claim 30 (depends on 1)

An e-mail architecture employing the system of claim 1.

### Claim 31 (depends on 1)

A computer readable medium having stored thereon the components of claim 1.

### Claim 32

A method that facilitates classifying messages in connection with spam prevention comprising:
receiving a set of the messages;
identifying intended recipients of the messages;
tagging a subset of the messages to be polled, the subset of messages corresponding to a subset of the recipients that are known spam fighting users;
receiving information relating to the user's classification of polling messages; and
employing the information in connection with training a spam filter and populating a spam list.

### Claim 33 (depends on 32)

The method of claim 32, wherein at least a subset of messages that would ordinarily be filtered are received by an email server and into a feedback loop system.

### Claim 34 (depends on 32)

The method of claim 32, wherein all incoming messages are handled by client email software such that messages selected for polling are specific to preferences of an individual user.

### Claim 35 (depends on 32)

The method of claim 32, wherein all messages received are considered for polling to mitigate bias of data.

### Claim 36 (depends on 32)

The method of claim 32, wherein the subset of messages to be polled comprises all messages.

### Claim 37 (depends on 32)

The method of claim 32, wherein the subset of recipients comprises all recipients.

### Claim 38 (depends on 32)

The method of claim 32, wherein the subset of recipients which are known spam fighting users is determined by each recipient performing at least one of the following:
opting in to provide feedback on messages to facilitate training a new spam filter;
passively opting in to provide feedback on messages by not opting out;
paying for email and message services provided by a participating message server; and
opening an email account with a participating message server.

### Claim 39 (depends on 32)

The method of claim 32, wherein the subset of users selected to participate in the message polling is selected at random.

### Claim 40 (depends on 32)

The method of claim 32, wherein the subset of users selected to participate in the message polling are selected from all paying users, thereby making it more expensive for some spammers to subvert the spam filter training.

### Claim 41 (depends on 32)

The method of claim 32, wherein the subset of messages tagged for polling are selected at random.

### Claim 42 (depends on 32)

The method of claim 32, wherein the subset of messages tagged for polling is limited by one or more polling limits.

### Claim 43 (depends on 32)

The method of claim 32, wherein the one or more polling limits comprises a per user limit and a per user per time period limit to mitigate bias of data.

### Claim 44 (depends on 32)

The method of claim 32, further comprising modifying tagged messages to mark and identify them as polling messages.

### Claim 45 (depends on 44)

The method of claim 44, wherein modifying the tagged messages comprises performing at least one of the following:
moving the tagged message to a separate folder for polling messages;
modifying the “from” address of the tagged message;
modifying the subject line of the tagged message;
using a polling icon on the tagged message to identify it as a polling message; and
using a unique color to identify the tagged message as a polling message.

### Claim 46 (depends on 32)

The method of claim 32, wherein the polling message comprises an attachment of the message as originally received and a set of instructions instructing the user on how to vote.

### Claim 47 (depends on 46)

The method of claim 46, further comprising at least two voting buttons to facilitate classifying the message as spam and not spam.

### Claim 48 (depends on 47)

The method of claim 47, wherein the voting buttons are links, which when selected by the user, create feedback that is employed in connection with training a spam filter and populating a spam list, the feedback comprising information relating to the selected classification, the user, the message, and a unique ID assigned to one of the message and contents of the message.

### Claim 49 (depends on 47)

The method of claim 47, further comprising a third voting button to opt out of future polling.

### Claim 50 (depends on 47)

The method of claim 47, wherein the voting buttons are incorporated into the polling message by modifying text of the message before sending the polling message to the respective user.

### Claim 51 (depends on 47)

The method of claim 47, wherein the voting buttons are implemented by modifying a user interface of client email software.

### Claim 52 (depends on 47)

The method of claim 47, wherein the voting buttons are incorporated into the polling message.

### Claim 53 (depends on 32)

The method of claim 32, further comprising scanning the tagged messages for viruses before they are downloaded for polling.

### Claim 54 (depends on 53)

The method of claim 53, further comprising removing the viruses from any infected messages.

### Claim 55 (depends on 53)

The method of claim 53, wherein tagged messages infected with a virus are discarded.

### Claim 56 (depends on 46)

The method of claim 46, further comprising a summary of the message, the summary comprising at least one of a subject line, message sender, date the message was sent, date the message was received, and a first few lines of text from the message.

### Claim 57 (depends on 32)

The method of claim 32, further comprising making a copy of each tagged message as originally received such that the respective users receive a first copy of the message in its original form and a second copy of the message in a form for polling.

### Claim 58 (depends on 32)

The method of claim 32, wherein the tagged messages are individually assigned a unique ID corresponding to at least one the tagged message and contents of the tagged message.

### Claim 59 (depends on 58)

The method of claim 58, wherein the tagged message and its associated ID are stored in a database in connection with training a spam filter and populating a spam list.

### Claim 60 (depends on 32)

The method of claim 32, wherein a feedback component receives the information relating to the user's classification of the polling message, the feedback component comprising a central database.

### Claim 61 (depends on 60)

The method of claim 60, wherein the database provides information in connection with training a spam filter and populating a spam list via a machine-learning technique.

### Claim 62 (depends on 32)

The method of claim 32, wherein the spam filter is trained using messages classified as spam and not spam to mitigate bias of polling data and misclassification of the polling messages.

### Claim 63 (depends on 32)

The method of claim 32, further comprising distributing the trained spam filter to one or more servers, the distribution occurring automatically and/or by request by at least one of an email message and a posting on a website for downloading.

### Claim 64 (depends on 60)

The method of claim 60, wherein identifying users and tagging messages for polling is distributed across one or more mail servers and one or more client email software such that data generated by the mail servers and client email software are returned to a central database for storage in connection with training a spam filter and populating a spam list.

### Claim 65 (depends on 64)

The method of claim 64, wherein key information is removed from any data that is sent to the central database by the mail servers and client email software for privacy reasons, such that only a portion of the data is sent to the central database to facilitate training the spam filter.

### Claim 66 (depends on 65)

The method of claim 65, wherein the portion of data sent to the central database comprises at least one of the following:
information relating to spam messages;
domain names embedded in legitimate messages; and
IP addresses embedded in legitimate messages.

### Claim 67 (depends on 64)

The method of claim 64, wherein the data generated by the mail servers and the data generated by the client email software are aggregated into statistical data, respectively, corresponding to polling results and polling messages, thus mitigating bandwidth required to transmit the data to the central database.

### Claim 68 (depends on 32)

The method of claim 32, wherein training the spam filter and populating the spam list is performed by machine learning techniques using data based on user classification feedback and optionally, data generated by one or more additional sources, the one or more sources comprising honeypots, recipient non-user classification feedback, and active learning techniques.

### Claim 69 (depends on 68)

The method of claim 68, wherein data generated by the one or more sources is re-weighted proportionately with respect to the type of data generated by the source and relative to the user classification data to facilitate obtaining an unbiased sampling of data.

### Claim 70 (depends on 68)

The method of claim 68, wherein honeypots correspond to email addresses disclosed in a restrictive manner such that it is known who is sending them legitimate messages, thereby facilitating immediate identification of spammers, verification of suspect merchants who distribute user subscriber information to spammers, and immediate classification of spam messages without waiting for user classification.

### Claim 71 (depends on 70)

The method of claim 70, wherein information generated by the honeypots is down weighted selectively depending at least in part on a number of honeypots in use relative to a number of other sources including user classification feedback.

### Claim 72 (depends on 70)

The method of claim 70, wherein data generated by the honeypots is integrated in real time into a central database, where information relating to user classifications and polling messages are also stored for later use in connection with training a spam filter and populating a spam list.

### Claim 73 (depends on 67)

The method of claim 67, wherein the messages are selected using active learning techniques, i.e. techniques that select messages based on their estimated value to learning new or updated filters.

### Claim 74 (depends on 32)

The method of claim 32 further comprising:
monitoring incoming messages for their respective one or more positive features;
determining a frequency of positive features received;
determining whether one or more positive features received exceeds a threshold frequency based at least in part upon historical data; and
quarantining suspicious messages, which correspond to the one or more positive features that exceed the threshold frequency, until further classification data is available to determine whether suspicious messages are spam.

### Claim 75

The method of claim,74 wherein the feature used is information about the sender comprising at least one of the sender's IP address and domain.

### Claim 76 (depends on 74)

The method of claim 74, wherein quarantining suspicious messages is performed by at least one of the following acts:
provisionally labeling the suspicious messages as spam and moving them to a spam folder;
delaying delivery of the suspicious messages to the user(s) until further classification data is available; and
storing the suspicious messages in a folder not visible to the user(s).

### Claim 77 (depends on 32)

The method of claim 32, further comprising determining false positive and catch rates of the spam filter to facilitate optimization of the spam filter, wherein determining false positive and catch rates comprises:
training the spam filter using a training data set, the training data set comprising a first set of polling results;
classifying a second set of polling messages using user feedback to yield a second set of polling results;
running the second set of polling messages through the trained spams filter;
comparing the second set of polling results to the trained spam filter results to determine false positive and catch rates of the filter to thereby evaluate and tune filter parameters according to optimal filter performance.

### Claim 78 (depends on 77)

The method of claim 77, wherein more than one spam filter is built, each having different parameters and each being trained on the same training data set, such that the false positive and catch rates of each spam filter is compared to at least one other spam filter to determine optimal parameters for spam filtering.

### Claim 79 (depends on 32)

The method of claim 32, further comprising building an improved spam filter using additional sets of incoming messages, subsets of which are subjected to polling to yield new information in connection with training the improved spam filter, wherein previously obtained information is re-weighted based at least in part upon how long ago it was obtained.

### Claim 80 (depends on 32)

The method of claim 32, further comprising employing the information to build a legitimate sender list.

### Claim 81 (depends on 80)

The method of claim 80, wherein the legitimate sender list comprises any one of IP addresses, domain names, and URLs that are substantially classified as sources of good mail according to a percentage of messages classified as good.

### Claim 82 (depends on 32)

The method of claim 32, wherein the spam lists are used to generate a black hole list of addresses from whom no mail would be accepted.

### Claim 83 (depends on 32)

The method of claim 32, further comprising employing the information to facilitate terminating accounts of spammers.

### Claim 84 (depends on 83)

The method of claim 83, further comprising identifying a spammer who is using an ISP and automatically notifying the ISP of the spamming.

### Claim 85 (depends on 83)

The method of claim 83, further comprising identifying a domain responsible for sending spam, and automatically notifying at least one of the domain's email provider and the domain's ISP of the spamming.

### Claim 86 (depends on 32)

The method of claim 32, further comprising distributing at least one of the spam filter and the spam list to any one of mail servers, email servers, and client email software, wherein distributing comprises at least one of the following:
posting a notification on a website notifying that the spam filter and spam list are available for downloading;
automatically pushing the spam filter and spam list out to mail servers, email servers and client email software; and
manually pushing the spam filter and spam list out to mail servers, email servers, and client email software.

### Claim 87

A cross-validation method that facilitates verifying reliability and trustworthiness of user classifications comprising:
excluding one or more suspected user's classifications from data employed to train the spam filter;
training the spam filter using all other available user classifications; and
running the suspected user's polling messages through the trained spam filter to determine how it would have classified the messages compared to the suspected user's classifications.

### Claim 88 (depends on 87)

The method of claim 87, further comprising performing at least one of the following:
discounting existing and future classifications provided by users who are determined to be untrustworthy until the users are determined to be trustworthy;
discarding existing classifications provided by users determined to be untrustworthy; and
removing the untrustworthy users from future polling.

### Claim 89

A method that facilitates verifying reliability and trustworthiness in user classifications for training a spam filter via a feedback loop system comprising:
identifying a subset of spam-fighting users as suspect users;
providing one or more messages having a known result to the suspect users for polling; and
determining whether the suspected users' classification of the one or more test messages matches the known classification to ascertain the reliability of the users' classifications.

### Claim 90 (depends on 89)

The method of claim 89, wherein the subset of spam-fighting users identified as suspect users comprises all users.

### Claim 91 (depends on 89)

The method of claim 89, wherein the message is a test message that is known to be at least one of spam and good mail and that is injected into a stream of incoming mail by the feedback loop system and delivered to the suspect users.

### Claim 92 (depends on 89)

The method of claim 89, wherein the message received by the suspected users for polling is hand-classified by a system administrator to train the spam filter with a correct classification to identify untrustworthy users.

### Claim 93 (depends on 89)

The method of claim 89, further comprising at least one of the following acts:
discounting existing and future classifications provided by users who are determined to be untrustworthy until the users are determined to be trustworthy;
discarding existing classifications provided by users determined to be untrustworthy; and
removing the untrustworthy users from future polling.

### Claim 94

A computer-readable medium having stored thereon the following computer executable components:
a component that receives a set of messages;
a component that identifies intended recipients of the messages, and tags a subset of the messages to be polled, the subset of messages corresponding to a subset of recipients that are known spam fighting users;
a message modification component that modifies the tagged messages to identify them as polling messages to users; and
a feedback component that receives information relating to the user's classification of the polled messages, and employs the information in connection with training a spam filter and populating a spam list.

### Claim 95

A system that facilitates classifying messages in connection with spam prevention comprising:
means for receiving a set of the messages;
means for identifying intended recipients of the messages;
means for tagging a subset of the messages to be polled, the subset of messages corresponding to a subset of the recipients that are known spam fighting users;
means for receiving information relating to the user's classification of the polling messages; and
means for employing the information in connection with training a spam filter and populating a spam list.
