US 2021/0263971 A1Published application

Automatic corpora annotation

AssigneeHealth Care Service Corporation, a Mutual Legal Reserve Company

InventorPaul Landes

Life of this patent

Estimating the term…

Loading patent…

Abstract

A computer implemented method and system for automatically creating an annotated dataset. An automatic annotating system may access a proprietary database and an unannotated dataset and identify tokens, or character spans, of the unannotated dataset that match property values in the database. The automatic annotating system may then determine whether the identified tokens in the unannotated dataset originated, or derived, from the database by calculating probabilities using a language model and a Bayesian network. The automatic annotating system annotates identified tokens determined to originate from the database by associating a tag to each identified token and assigning annotation attributes for each tag. The annotations and associated properties and values are stored as an annotated dataset. The annotated dataset may then be used train automated, machine learned models to identify and tag other datasets.

Claims20

Dependent claims

A computer implemented method comprising:

accessing, by a processor, a database stored in a memory, the database comprising a plurality of data items, each data item comprising one or more properties, each property of the one or more properties having an associated value, the database being structured with a pre-defined data model or format;
accessing, by the processor, a first dataset stored in the memory and comprising text, wherein a portion of the text contains data derived from the database;
segmenting, by the processor, the text of the first dataset into tokens, the tokens comprising one or more characters;
identifying, by the processor, tokens in the first dataset that match property values in the database for predetermined database properties;
determining, by the processor, whether the identified tokens in the first dataset represent values associated with a property in the database;
annotating, by the processor, the identified tokens of the first dataset when the identified tokens are determined to represent values associated with a property in the database, wherein annotating comprises associating a tag with each identified token and assigning annotation attributes for each tag;
and
storing, by the processor, the annotations and associated database properties and database values in the memory as an annotated dataset.

5 more elements

An automatic annotating system comprising:

a data preparer configured to access, from an authorized system, a database and a first dataset stored in a memory, the database comprising a plurality of data items, each data item comprising one or more properties, each property of the one or more properties having an associated value, the first dataset comprising text, wherein a portion of the text contains data derived from the database;
a tokenizer coupled with the data preparer and configured to segment the text of the first dataset into tokens, the tokens comprising one or more characters;
a data analyzer coupled with the tokenizer and configured to identify tokens in the first dataset that match property values in the database for predetermined database properties and determine whether the identified tokens in the first dataset represent values associated with a property in the database;
and
an annotator coupled with the data analyzer and configured to annotate the identified tokens of the first dataset when the identified tokens are determined to represent values associated with a property in the database, wherein the annotator, to annotate the identified tokens, is further configured to associate a tag with each identified token and assign annotation attributes for each tag,
wherein the respective tags, the identified tokens associated with the respective tags, and the assigned annotation attributes for the respective tags are stored in the memory as an annotated dataset.

3 more elements

Dependencies are inferred from the claim text.