Method of feature extraction from noisy documents
AssigneeXerox Corporation
InventorsLoic Lecerf, Boris Chidlovskii
Life of this patent
Estimating the term…
Abstract
Aspect of the exemplary embodiment relate to a method and apparatus for automatically identifying features that are suitable for use by a classifier in assigning class labels to text sequences extracted from noisy documents. The exemplary method includes receiving a dataset of text sequences, automatically identifying a set of patterns in the text sequences, and filtering the patterns to generate a set of features. The filtering includes at least one of filtering out redundant patterns and filtering out irrelevant patterns. The method further includes outputting at least some of the features in the set of features, optionally after fusing features which are determined not to affect the classifiers accuracy if they are merged.
Claims20
A computer-implemented method comprising:
- receiving a dataset of text sequences extracted from document images;
- automatically identifying a set of patterns in the text sequences;
- filtering the patterns to generate a set of features, the filtering comprising at least one of:
- filtering out redundant patterns, and
- filtering out irrelevant patterns;
- and
- outputting at least some of the features in the set of features.
An apparatus for generating a set of features comprising:
- a frequent pattern extractor configured for extracting frequent patterns from an input dataset of extracted text sequences;
- a feature selection module configured for identifying features from among the frequent patterns by filtering out at least one of:
- patterns which are not relevant, and patterns which are redundant;
- and
- optionally, a feature fusion module configured for identifying features which are able to be merged.
A method for training a classifier comprising:
- automatically extracting frequent patterns from an input dataset of text sequences without regard to class labels of the text sequences;
- identifying features from among the frequent patterns by filtering out non-relevant and redundant patterns;
- and
- optionally applying at least one criterion for determining whether a plurality of the features are able to be merged and if the at least one criterion is met, merging those features;
- inputting the features, optionally after merging of features, into the classifier;
- and
- training the classifier based on the input features and class labels assigned to the corresponding text sequences.
Dependencies are inferred from the claim text.