# US 2009/0276381 A1 · Social knowledge system content quality

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2009/0276381 A1
- Document ID: US20090276381A1
- Status: Published application
- Kind: A1
- Application number: 12/491,133
- Filed: 2009-06-24
- Published: 2009-11-05
- Earliest claimed: 2006-10-18
- Assignees: Yahoo! Inc.
- Inventors: Daniel Boies; Benoit Dumoulin; Remi Kwan
- Web page: https://patentgrep.com/patent/US20090276381A1

## CPC

- G06N 5/022: Knowledge engineering; Knowledge acquisition
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Techniques for automatically scoring submissions to an online question-and-answer submission system are disclosed. According to one such technique, an initial set of user submissions are scored by human operators and/or automated algorithmic mechanisms. The submissions and their accompanying scores are provided as training data to an automated machine learning mechanism. The machine learning mechanism processes the training data and automatically detects patterns in the provided submissions. The machine learning mechanism automatically correlates these patterns with the scores assigned to the submissions that match those patterns. As a result, the machine learning mechanism is trained. Thereafter, the machine learning mechanism processes unscored submissions. The machine learning mechanism automatically identifies, from among the patterns that the machine learning mechanism has already detected, one or more patterns that these submissions match. The machine learning mechanism automatically scores these submissions based on the matching patterns and the scores that are associated with those patterns.

## Claims

### Claim 1

A computer-implemented method comprising steps of:
receiving one or more answer submissions at an online answer submission system that accepts, from multiple users, answers to questions submitted to the online answer submission system by users other than those that submitted the one or more answer submissions;
processing a set of previously scored training submissions, thereby training a machine learning mechanism to score, automatically, a plurality of submissions that are submitted by users of a system;
scoring a particular submission of said plurality of submissions automatically using the machine learning mechanism, thereby producing a score; and
performing, relative to the particular submission, an action that is determined based on said score;
wherein said previously scored training submissions are also answers to questions submitted to the online answer submission system;
wherein said steps are performed by one or more computing devices.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
based on said score, automatically limiting, within an online submission system, privileges of a submitter that submitted the particular submission.

### Claim 3 (depends on 1)

The method of claim 1, wherein processing the set of previously scored training submissions comprises automatically identifying, in the previously scored training submissions, co-occurrences of words that are correlated with the scores that one or more of the previously scored training submissions received.

### Claim 4 (depends on 1)

The method of claim 1, wherein processing the set of previously scored training submissions comprises automatically identifying, in the previously scored training submissions, abbreviated phrases are correlated with the scores that one or more of the previously scored training submissions received.

### Claim 5 (depends on 1)

The method of claim 1, wherein processing the set of previously scored training submissions comprises automatically identifying, in the previously scored training submissions, grammatical errors that are correlated with the scores that one or more of the previously scored training submissions received.

### Claim 6 (depends on 1)

The method of claim 1, wherein processing the set of previously scored training submissions comprises automatically identifying one or more behavioral characteristics of submitters of the previously scored training submissions, wherein identifying the one or more behavioral characteristics comprises identifying one or more behavioral characteristics that are correlated with the scores that one or more of the previously scored training submissions received.

### Claim 7 (depends on 6)

The method of claim 6, wherein identifying the one or more behavioral characteristics of submitters of the previously scored training submissions comprises automatically identifying, in Internet browsing histories of submitters of the previously scored training submissions, characteristics that are correlated with the scores that one or more of the previously scored training submissions received.

### Claim 8 (depends on 1)

The method of claim 1, wherein processing the set of previously scored training submissions comprises automatically identifying question-asker-assigned ratings of one or more of the previously scored training submissions, wherein identifying the question-asker-assigned ratings comprises identifying question-asker-assigned ratings that are correlated with the scores that one or more of the previously scored training submissions received.

### Claim 9 (depends on 1)

The method of claim 1, wherein performing the action based on the score comprises adjusting, based on the score, an order in which the particular submission will appear within an ordered list of submissions.

### Claim 10 (depends on 1)

The method of claim 1, further comprising:
based on one or more scores that have been assigned to one or more submissions submitted by a particular submitter, classifying the particular submitter with a particular classification; and
based on the particular submitter being classified with the particular classification, presenting, to the particular submitter, one or more questions to which the submitter can submit answers.

### Claim 11 (depends on 1)

The method of claim 1, wherein training the machine learning mechanism comprises training the machine learning mechanism based on at least one of: (a) character distribution entropy in a submission of the plurality of submissions, (b) word distribution entropy in a submission of the plurality of submissions, (c) word length distribution entropy in a submission of the plurality of submissions, (d) a length of a submission of the plurality of submissions, (e) lexical distance between submission text that contains misspellings and corresponding reference text that does not contain misspellings, and (f) a number of words in a submission of the plurality of submissions that are absent from a dictionary of known words for a specific language.

### Claim 12 (depends on 1)

The method of claim 1, wherein training the machine learning mechanism comprises training the machine learning mechanism based on web-based sources of information, wherein the web-based sources of information comprise at least one of: (a) information that is extracted from an analysis of search results that result from a web search query, (b) a number of trusted web sites within a specified set of search results, (c) information produced by a lexical analysis of text that corresponds to a specified set of search results, (d) and information produced from an expansion of a submission with lexical information that originates from a specified set of search results.

### Claim 13 (depends on 1)

The method of claim 1, wherein training the machine learning mechanism comprises training the machine learning mechanism based on information that was extracted from a search engine's analysis of search results that resulted from a web search query, wherein the information comprises at least one of: (a) a number of sites occurring within a specified set of search results that have been classified as adult sites, and (b) a number of links within a specified set of search results that have been classified as spam links.

### Claim 14

A volatile or non-volatile computer-readable medium that stores instructions which, when executed by one or more processors, cause the one or more processors to perform steps comprising:
receiving one or more answer submissions at an online answer submission system that accepts, from multiple users, answers to questions submitted to the online answer submission system by users other than those that submitted the one or more answer submissions;
processing a set of previously scored training submissions, thereby training a machine learning mechanism to score, automatically, a plurality of submissions that are submitted by users of a system;
scoring a particular submission of said plurality of submissions automatically using the machine learning mechanism, thereby producing a score; and
performing, relative to the particular submission, an action that is determined based on said score;
wherein said previously scored training submissions are also answers to questions submitted to the online answer submission system.
