Techniques for assigning labels to dataset fields
AssigneeAb Initio Technology LLC
InventorsSandrick Melbouci, Andrew Schon
Life of this patent
Estimating the term…
Abstract
Techniques for processing a dataset comprising data stored in fields to identify field labels. The field labels describe data stored in the dataset fields. The techniques determine whether any field labels in a field label glossary match a field. If none of the field labels in the field label glossary match the field, the techniques generate a new field label using the name of the field. The generated field label may be assigned to the field.
Claims20
A method for processing a dataset comprising data stored in fields to identify, from a field label glossary, a field label for each field in a set of one or more of the dataset fields of the dataset, the field labels describing data stored in the set of fields, the method comprising:
- using at least one computer hardware processor to perform:
- for each particular field in the set of fields,
- determining, using a name of the particular field and natural language processing (NLP), a first set of candidate field labels for the particular field and field name analysis scores for the first set of candidate field labels;
- determining, using a subset of data from the particular field and tests associated with respective field labels in the field label glossary, a second set of candidate field labels and field data analysis scores for the second set of candidate field labels;
- determining merged candidate field labels and corresponding scores using the first set of candidate field labels and the field name analysis scores, and the second set of candidate field labels and the field data analysis scores;
- and
- assigning one of the merged candidate field labels to the particular field using the corresponding scores.
A system for processing a dataset comprising data stored in fields to identify, from a field label glossary, a field label for each field in a set of one or more of the dataset fields of the dataset, the field labels describing data stored in the set of fields, the system comprising:
- at least one computer hardware processor;
- and
- at least one non-transitory computer-readable storage medium storing instructions that, when executed by the at least one computer hardware processor, cause the at least one computer hardware processor to perform:
- for each particular field in the set of fields,
- determining, using a name of the particular field and natural language processing (NLP), a first set of candidate field labels for the particular field and field name analysis scores for the first set of candidate field labels;
- determining, using a subset of data from the particular field and tests associated with respective field labels in the field label glossary, a second set of candidate field labels and field data analysis scores for the second set of candidate field labels;
- determining merged candidate field labels and corresponding scores using the first set of candidate field labels and the field name analysis scores, and the second set of candidate field labels and the field data analysis scores;
- and
- assigning one of the merged candidate field labels to the particular field using the corresponding scores.
A non-transitory computer-readable storage medium storing instructions that, when executed by at least one computer hardware processor, cause the at least one computer hardware processor to perform a method for processing a dataset comprising data stored in fields to identify, from a field label glossary, a field label for each field in a set of one or more of the dataset fields of the dataset, the field labels describing data stored in the set of fields, the method comprising:
- for each particular field in the set of fields,
- determining, using a name of the particular field and natural language processing (NLP), a first set of candidate field labels for the particular field and field name analysis scores for the first set of candidate field labels;
- determining, using a subset of data from the particular field and tests associated with respective field labels in the field label glossary, a second set of candidate field labels and field data analysis scores for the second set of candidate field labels;
- determining merged candidate field labels and corresponding scores using the first set of candidate field labels and the field name analysis scores, and the second set of candidate field labels and the field data analysis scores;
- and
- assigning one of the merged candidate field labels to the particular field using the corresponding scores.
Dependencies are inferred from the claim text.