Method and apparatus for document classification from degraded images
AssigneesXerox Corporation, Fuji Xerox Co., Ltd.
InventorTakehiro Nakayama
Life of this patent
Estimating the term…
Abstract
A word shape token-based document classification system prepares a plurality of sets of training data degraded by image quality and selects the optimum training data set by examining scores from a relevance measurement. The system achieves high accuracy from a wide range of image quality.
Claims22
A method for classifying a document using a sequence of word tokens representing words in the document, the word tokens comprising character shape code classes, each word of the document being represented by only one word token, and a plurality of language training data sets, each language training data set derived from an image of language reference data and comprising a plurality of language profiles, the method comprising the steps:
- merging, for each of the plurality of language training data sets, the language profiles of that language training data set to form a merged language profile;
- determining the relative frequencies of word shape tokens for each merged language profile and the document;
- selecting the language training data set corresponding to the merged language profile having relative frequencies that are the most similar to the relative frequencies of the document;
- determining the relative frequencies of word shape tokens for each language profile in the selected language training data set;
- selecting the language profile having relative frequencies that are the most similar to the document;
- and
- classifying the document as being composed in the language of the selected language profile, wherein at least one of the language training data sets is derived from a degraded image of the language reference data.
A method for classifying a document using a sequence of word tokens representing words in the document, the word tokens comprising character shape code classes, each word of the document being represented by only one word token, and a plurality of language training data sets, each language training data set derived from an image of language reference data and comprising a plurality of language profiles, the method comprising the steps of;
- determining the relative frequencies of word shape tokens for each language profile in each language training data set and the document;
- determining a language score for each language profile based upon the degree of similarity between the relative frequencies of each language profile and the document;
- selecting the language profile having the highest language score;
- and
- classifying the document as being composed in the language of the selected language profile, wherein at least one of the language training data sets is derived from a degraded image of the language reference data set.
An apparatus for classifying a document using a sequence of word tokens representing words in the document, the word tokens comprising character shape code classes, each word of the document being represented by only one word token, and a plurality of language training data sets, each language training data set derived from an image of language reference data and comprising a plurality of language profiles, the apparatus comprising:
- a language profile merging circuit that merges all language profiles in each of the plurality of language training data sets to form a merged language profile for each language training data set;
- a first word shape token relative frequency determining circuit that determines the relative frequencies of word shape tokens for each merged language profile and the document;
- a language training data set selector that selects a language training data set corresponding to the merged language profile having relative frequencies that are the most similar to the relative frequencies of the document;
- a second word shape token relative frequency determining circuit that determines the relative frequencies of word shape tokens for each language profile in the selected language training set;
- a language profile selector that selects the language profile having relative frequencies that are the most similar to the document;
- and
- a document language classifier that classifies the document as being composed in the language of the selected language profile, wherein at least one of the language training data sets is derived from a degraded image of the language reference data set.
An apparatus for classifying a document using a sequence of word tokens representing words in the document, the word tokens comprising character shape code classes, each word of the document being represented by only one word token, and a plurality of language training data sets, each language training data set derived from an image of language reference data and comprising a plurality of language profiles, the apparatus comprising:
- a word shape token relative frequency determining circuit that determines the relative frequencies of word shape tokens for each language profile in each language training set and the document;
- a language score determining circuit that determines a language score for each language profile based upon the degree of similarity between the relative frequencies of each language profile and the document;
- a language profile selecting circuit that selects the language profile having the highest language score;
- and
- a document language classifying circuit that classifies the document as being composed in the language of the selected language profile, wherein at least one of the language training data sets is derived from a degraded image of the language reference data set.
Dependencies are inferred from the claim text.