Information extraction and annotation systems and methods for documents
AssigneeOpen Text Holdings, Inc.
InventorsJulian Markus Riediger, Andy Horng
Life of this patent
Estimating the term…
Abstract
Information extraction and annotation systems and methods for use in annotating and determining annotation instances are provided herein. Exemplary methods include receiving training documents having annotated words, identifying a predetermined number of characters preceding and following each annotated word for each of the training documents to determine a context for each of the annotated words, performing an alignment of an annotated word and its context with characters in the target document, identifying common sequences, and assigning annotations to words in the target document when common sequences are found.
Claims9
A method for annotating documents using a context and contiguity analysis module having a processor and a memory for storing logic, the processor executing the logic to perform the method, comprising:
- receiving training documents, the training documents having annotated words;
- identifying a predetermined number of characters preceding and following each annotated word for each of the training documents to determine a context for each of the annotated words;
- performing an alignment of an annotated word and its context with characters in a target document;
- identifying common sequences between the target document and a training corpus, based upon the alignment;
- dividing the target document into n-gram strings;
- identifying overlapping sequences of characters between the n-gram strings of the target document and the training documents;
- assigning an annotation to a word in the target document when the overlapping sequences between the n-gram strings of the target document and the training documents meet or exceed a threshold;
- for each annotated word in the training documents, determining a distance between the annotated word and one or more document features;
- calculating an averaged location for each annotated word in the training documents, the averaged location defining an averaged placement of each annotated word within the training documents;
- identifying matches between a distance and averaged location of words of the target document and distances and the averaged location of at least one of the annotated words of the training documents;
- and
- based at least in part on the identified matches, assigning an annotation to a word of the target document that matches the at least one of the annotated words of the training documents.
A method for annotating documents using a context and contiguity analysis module having a processor and a memory for storing logic, the processor executing the logic to perform the method, comprising:
- receiving training documents, the training documents having annotated words;
- identifying a predetermined number of characters preceding and following each annotated word for each of the training documents to determine a context for each of the annotated words;
- dividing a target document into n-gram strings;
- identifying overlapping sequences of characters between the n-gram strings of the target document and the training documents;
- assigning an annotation to each word in the target document when the overlapping sequences between the n-gram strings of the target document and the training documents meet or exceed a threshold;
- for each annotated word in the training documents, determining a distance between the annotated word and one or more document features;
- calculating an averaged location for each annotated word in the training documents, the averaged location defining an averaged placement of each annotated word within the training documents;
- identifying matches between a distance and averaged location of words of the target document and distances and the averaged location of at least one of the annotated words of the training documents;
- and
- based at least in part on the identified matches, assigning an annotation to a word of the target document that matches the at least one of the annotated words of the training documents.
A context and contiguity analysis module, comprising:
- a processor;
- and
- a memory for storing logic, the processor executing the logic to:
- receive training documents, the training documents having annotated words;
- identify a predetermined number of characters preceding and following each annotated word for each of the training documents to determine a context for each of the annotated words;
- perform an alignment of an annotated word and its context with characters in a target document, by dividing the target document into n-gram strings and identifying overlapping sequences of characters between the n-gram strings of the target document and the training documents,
- wherein the target document comprises a document that has been processed using optical character recognition (OCR), the document comprising OCR errors that distort at least a portion of the characters of the target document;
- for each annotated word in the training documents, determine a distance between the annotated word and one or more document features;
- calculate an averaged location for each annotated word in the training documents, the averaged location defining an averaged placement of each annotated word within the training documents;
- identify matches between a distance and averaged location of words of the target document and distances and the averaged location of at least one of the annotated words of the training documents;
- and
- in response to performing the alignment, assign an annotation to a word in the target document when a common sequence between the target document and the training documents is found, the annotation comprising an annotation of an annotated word of the training documents included in the common sequence.
Dependencies are inferred from the claim text.