US 11,886,820 B2 · System and method for machine-learning based extraction of information from documents

Abstract

A method and system are provided for training a machine-learning (ML) system/module and to provide an ML model. In one embodiment, a method includes using a labeled entities set to train a machine learning (ML) system, to obtain an ML model, and using the trained ML model to predict labels for entities in an unlabeled entities set, yielding a machine-labeled entities set. One or more individual ML models may be trained and used in this way, where each individual ML model corresponds to a respective document source. The document sources can be identified via classification of a corpus of documents. The prediction of labels provides a respective confidence score for each machine-labeled entity. The method also includes selecting from the machine-labeled entities set, a subset of machine-labeled entities having a respective confidence score at least equal to a threshold confidence score; and updating the labeled entities set by adding thereto the selected subset of machine-labeled entities. The method further includes removing from the machine-labeled entities set the selected subset of machine-labeled entities and deleting labels assigned to the entities in the updated machine-labeled entities set to provide the unlabeled entities set for a next iteration. The method also includes, if a termination condition is not reached, repeating the steps above and, otherwise, storing the ML model.

Claims20

Dependent claims

A method for training a machine-learning (ML) system, the method comprising:

(a) providing a seed set of labeled entities as a labeled entities set based on a first cluster of a plurality of clusters of documents and using the labeled entities set to train the ML system, to obtain an ML model;
(b) using the trained ML system to predict labels for entities in an unlabeled entities set, yielding a machine-labeled entities set, the prediction providing a respective confidence score for each machine-labeled entity;
(c) selecting from the machine-labeled entities set, a subset of machine-labeled entities having a respective confidence score at least equal to a threshold confidence score;
(d) updating the labeled entities set by adding thereto the selected subset of machine- labeled entities;
(e) removing from the machine-labeled entities set the selected subset of machine-labeled entities and deleting labels assigned to the entities in the updated machine-labeled entities set to provide the unlabeled entities set for a next iteration;
(f) if a termination condition is not reached, repeating steps (a) through (e), and, otherwise, storing the ML model;
(g) selecting a second cluster from the plurality of clusters;
and
(h) repeating the steps (a) through (f) for the second cluster to store a different ML model for the second cluster, wherein providing the seed set in step (a) is based on the second cluster.

6 more elements

A system for providing a machine-learning (ML) model, the system comprising:

a processor;
and
a memory in communication with the processor and comprising instructions which, when executed by the processor, program the processor to:
(a) provide a seed set of labeled entities as a labeled entities set based on a first cluster of a plurality of clusters of documents and use the labeled entities set to train a machine learning (ML) module, to obtain the ML model;
(b) use the trained ML module to predict labels for entities in an unlabeled entities set, yielding a machine-labeled entities set, the prediction providing a respective confidence score for each machine-labeled entity;
(c) select from the machine-labeled entities set, a subset of machine-labeled entities having a respective confidence score at least equal to a threshold confidence score;
(d) update the labeled entities set by adding thereto the selected subset of machine-labeled entities;
(e) remove from the machine-labeled entities set the selected subset of machine-labeled entities and delete labels assigned to the entities in the updated machine-labeled entities set to provide the unlabeled entities set for a next iteration;
(f) if a termination condition is not reached, repeat operations (a) through (e) and, otherwise, store the ML model;
(g) select a second cluster from the plurality of clusters;
and
(h) repeat the steps (a) through (f) for the second cluster to store a different ML model for the second cluster, wherein providing the seed set in step (a) is based on the second cluster.

9 more elements

Dependencies are inferred from the claim text.

System and method for machine-learning based extraction of information from documents

Life of this patent

System and method for machine-learning based extraction of information from documents

Life of this patent

Abstract

Claims20