US 8,554,715 B2Granted

Electronic document classification apparatus

AssigneeNederlandse Organisatie voor toegepast-natuurwetenschappelijk onderzoek TNO

InventorsWessel Kraaij, Stephan Alexander Raaijmakers

Life of this patent

Estimating the term…

Loading patent…

Abstract

The apparatus computes classification scores based on parameters that have been determined from documents. Each score is compared with a first and second threshold. Definite classifications are assigned when the score is above the highest threshold or below the lowest threshold and the documents are processed accordingly. If the score is between the thresholds the document is singled out for further inspection, for example by a human arbitrator, to assign a class. The first and second threshold are adapted automatically based on specified a minimum accuracy level for the classification and a training set. The apparatus uses this specified accuracy in a search for a combination of threshold values that optimizes classifier yield, in terms of a maximized fraction of patterns in a training set that need not be turned over for further inspection without definite classification. The search is subject to the condition that the combination of thresholds results in at least the specified accuracy over the training set.

Claims8

Dependent claims

A document classification apparatus, comprising

a processor;
a score computation module configured to compute a score value from information obtained for a document to be classified;
a threshold comparator configured to perform a comparison to determine whether the score value is above a first threshold, below a second threshold or between the first and second threshold;
a further processing module configured to process the document dependent on the result of the comparison, the further processing module being configured to single out the document for further inspection to assign a class if the score value is between the first and second threshold;
a threshold determination module configured to determine the effective levels of the first and second threshold, based on an input minimum accuracy value and a set of training examples, by searching for a combination of effective thresholds levels that maximizes a yield over the training set while at least achieving said input minimum accuracy value, wherein the threshold determination module is configured to determine the yield according to
yield=(TP+TN+FP+FN)/(TP+TN+FP+FN+M)
and the accuracy according to a function of TP and TN and/or FP and FN that increases with increasing TP and TN and/or decreases with increasing FP and FN, wherein TP and FP are counts of training patterns with a score value above the first threshold that are identified in the training pattern to belongs to a class and not to belong to the class respectively, wherein FN and TN are counts of training patterns with a score value below the second threshold that are identified in the training pattern to belongs to the class and not to belong to the class respectively, and M is a count of training patterns with a score value between the first and second threshold.

4 more elements

A computer implemented classification method, the method comprising

computing a score value from information obtained for an object to be classified;
determining whether the score value is above a first threshold, below a second threshold or between the first and second threshold;
applying further processing to the object dependent on the result of the comparison, singling out the object for further inspection to assign a class if the score value is between the first and second threshold;
determining effective levels of the first and second threshold based on an input minimum accuracy value and a set of training examples, by searching for a combination of effective threshold levels that maximizes a yield over the training set while at least achieving said input minimum accuracy value, the method comprising
counting template patterns to determine, for possible combinations of first and second effective threshold levels in the search, count values TP, FP, FN, TN and M, wherein TP and FP of training patterns with a score value above the first threshold that are identified in the training pattern to belongs to a class and not to belong to the class respectively, wherein FN and TN are counts of training patterns with a score value below the second threshold that are identified in the training pattern to belongs to the class and not to belong to the class respectively, and M is a count of training patterns with a score value between the first and second threshold, and
computing the yield in the search according to
yield=(TP+TN+FP+FN)/(TP+TN+FP+FN+M)
and the accuracy according to a function of TP and TN and/or FP and FN that increases with increasing TP and TN and/or decreases with increasing FP and FN.

5 more elements

Dependencies are inferred from the claim text.