Classifying documents using multiple classifiers

A non-transitory computer storage medium encoded with a computer program, the program comprising instructions that when executed by data processing apparatus cause the data processing apparatus to perform operations comprising:

• selecting a collection of documents to classify;
• selecting a plurality of classifiers for scoring a document with respect to a specified property;
• for each document in the collection,
• applying each of the plurality of classifiers, each classifier generating a score associated with a likelihood that the document has the specified property,
• combining the scores from each classifier including applying a multiple classifier model;
• and
• classifying the document as having the specified property based on the combined score;
• where the multiple classifier model is generated by:
• identifying a training group of documents;
• calculating classifier scores for each document in the training group of documents using the plurality of classifiers;
• determining whether each document in the training group of documents has the specified property;
• and
• generating a maximum likelihood estimate for a probability distribution by applying monotonic regression to classifier scores of each document in the training group of documents, where each classifier generates classifier scores of a document that monotonically increase with an increase in the likelihood that the document has the specified property.

A system comprising:

• one or more computers operable to interact with a device and to perform operations including:
• selecting a collection of documents to classify;
• selecting a plurality of classifiers for scoring a document with respect to a specified property;
• for each document in the collection,
• applying each of the plurality of classifiers, each classifier generating a score associated with a likelihood that the document has the specified property,
• combining the scores from each classifier including applying a multiple classifier model, and
• classifying the document as having the specified property based on the combined score;
• where the multiple classifier model is generated by:
• identifying a training group of documents;
• calculating classifier scores for each document in the training group of documents using the plurality of classifiers;
• determining whether each document in the training group of documents has the specified property;
• and
• generating a maximum likelihood estimate for a probability distribution by applying monotonic regression to classifier scores of each document in the training group of documents, where each classifier generates classifier scores of a document that monotonically increase with an increase in the likelihood that the document has the specified property.