Method and system for mapping labels in standardized tables using machine learning
AssigneeFactset Research System Inc.
InventorsYan Chen, Agrima Srivastava, Dakshina Murthy Malladi
Life of this patent
Estimating the term…
Abstract
A method and system for mapping labels of documents is described. A training set including a plurality of documents and at least one map can be retrieved. Each document can include a plurality of labels, and the at least one map can represent associations between the labels of one document and another document in the set. Each document (or group of documents) in the set can include certain features. These features can relate to the labels in the documents. Each label can correspond to one or more data points (or datasets) in each documents. In one example embodiment, the map can be generated based on the features extracted from each document.
Claims14
A method comprising:
- receiving a training set of documents, each document including a plurality of first labels associated with first data points, wherein each document displays the plurality of first labels and the first data points in a table;
- receiving a map demonstrating associations between respective first labels of the documents;
- extracting a first feature of each document in the training set, wherein the first feature is a similarity of a first position of at least one of the plurality of first labels in a first document to a second position of at least one of the plurality of first labels in a second document;
- extracting a second feature of each document in the training set, wherein the second feature is another similarity of at least one of the plurality of first labels appearing before or after at least another one of the plurality of first labels in the first document to at least one of the plurality of first labels appearing before or after at least another one of the plurality of first labels in the second document;
- training a classification model using the training set of documents, the map, the first feature, and the second feature;
- receiving a second set of documents, each document including a plurality of second labels associated with second data points;
- extracting a third feature of each document in the second set;
- providing the second set of documents and the third feature to the classification model;
- and
- receiving a prediction score from the classification model, the prediction score indicating a likelihood of two second labels being associated with each other.
Dependencies are inferred from the claim text.