Method and System for Characterizing Unknown Annotator and its Type System with Respect to Reference Annotation Types and Associated Reference Taxonomy Nodes

A system for characterizing a subject annotator according to a reference annotation type system, comprising:

• a machine-readable storage medium;
• a machine-readable code, stored on said machine-readable storage media, having instructions for the machine to apply a plurality of document annotators;
• a machine-readable set of reference annotation type systems, stored on said machine-readable storage medium;
• providing a machine-readable plurality of documents, stored on said machine-readable storage medium;
• a machine-readable code, stored on said machine-readable storage media, having instructions for annotating each of said plurality of documents using at least on of said reference set of document annotators and said reference annotation type systems to generate a pre-annotated reference document set;
• a machine-readable code, stored on said machine-readable storage media, having instructions for annotating said document corpus using said subject annotator and said subject annotation type system to generate a pre-annotated evaluation document set;
• a machine-readable code, stored on said machine-readable storage media, having instructions for comparing at least one of said documents in said pre-annotated evaluation document set to its corresponding documents in said pre-annotated reference document set, to generate a matching data representing matches in location, within said compared documents, between instances of annotations in the subject annotation type system and instances of annotations in the reference annotation type systems;
• and
• a machine-readable code, stored on said machine-readable storage media, having instructions for selecting, based on said matching data, a reference document annotation type system that meets a pre-determined correlation criterion with respect to said subject annotation type system.