Out of distribution element detection for information extraction

A system comprising:

• a set of processors;
• a memory storing a trained machine learning (“ML”) model, wherein the trained ML model is trained to classify elements in a document as belonging to one of a set of one or more in-distribution (“in-D”) class or to an out-of-distribution (“OO-D”) class, wherein training the trained ML model comprises:
• accessing a first training dataset, the first training dataset comprising a first plurality of documents and annotation information for each document in the first plurality of documents, wherein, for each document in the first plurality of documents, the annotation information for the document comprises information indicative of one or more elements in the document, and for each element in the one or more elements, information indicative of an in-D class to which the element belongs;
• generating a second training dataset based upon the first training dataset, the second training dataset comprising a second plurality of documents and annotation information for each document in the second plurality of documents, wherein, each document in the second plurality of documents includes one or more elements that belong to the OO-D class;
• and
• training the ML model using both the first training dataset and the second training dataset;
• wherein one or more processors from the set of processors are configured to perform processing comprising:
• classifying, using the trained ML model, a first element extracted from a document as belonging to an in-D class from the set of one or more in-D classes;
• and
• classifying, using the trained ML model, a second extracted element from the document as belonging to the OO-D class.

A non-transitory computer-readable medium storing computer-executable instructions that, when executed by one or more computer devices, cause the computing devices to perform processing comprising:

• accessing a trained machine learning (“ML”) model, wherein the trained ML model is trained to classify elements in a document as belonging to one of a set of one or more in-distribution (“in-D”) class or to an out-of-distribution (“OO-D”) class, wherein training the trained ML model comprises:
• accessing a first training dataset, the first training dataset comprising a first plurality of documents and annotation information for each document in the first plurality of documents, wherein, for each document in the first plurality of documents, the annotation information for the document comprises information indicative of one or more elements in the document, and for each element in the one or more elements, information indicative of an in-D class to which the element belongs;
• generating a second training dataset based upon the first training dataset, the second training dataset comprising a second plurality of documents and annotation information for each document in the second plurality of documents, wherein, each document in the second plurality of documents includes one or more elements that belong to the OO-D class;
• and
• training the ML model using both the first training dataset and the second training dataset;
• wherein one or more processors from the set of processors are configured to perform processing comprising:
• classifying, using the trained ML model, a first element extracted from a document as belonging to an in-D class from the set of one or more in-D classes;
• and
• classifying, using the trained ML model, a second extracted element from the document as belonging to the OO-D class.