Automated machine-learning dataset preparation
AssigneeInternational Business Machines Corporation
InventorsWillie Robert Patten, Jr., Eugene Irving Kelton, Arvin Bhatnagar, Jason Howard Cornpropst, Jacob McPherson
Life of this patent
Estimating the term…
Abstract
A method of preparing a dataset may comprise calculating a pattern relevance for a first field in the dataset and modifying the first field based on the pattern relevance. The method may further comprise detecting a contextual cue in the first field. The method may further comprise retrieving contextual information for a value in the first field and adding that contextual information to the database. Finally, the method may further comprise identifying a numerical scheme for the first field and parsing the first field into a number according to that numerical scheme.
Claims20
A method of preparing a dataset for ingestion by a machine-learning model, the method comprising:
- calculating a pattern relevance for a first field in the dataset;
- modifying the first field based on the pattern relevance;
- detecting a contextual cue in the first field;
- retrieving contextual information for a value in the first field;
- adding the contextual information to the dataset;
- identifying a numerical scheme for the first field;
- and
- parsing the first field into a number according to the numerical scheme.
A system comprising:
- a processor;
- and
- a memory in communication with the processor, the memory containing program instructions that, when executed by the processor, are configured to cause the processor to perform a method of preparing a dataset for ingestion by a machine-learning model, the method comprising:
- calculating a pattern relevance for a first field in the dataset;
- modifying the first field based on the pattern relevance;
- detecting a contextual cue in the first field;
- retrieving contextual information for a value in the first field;
- adding the contextual information to the dataset;
- identifying a numerical scheme for the first field;
- and
- parsing the first field into a number according to the numerical scheme.
A computer program product, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to:
- calculate a pattern relevance for a first field in a dataset;
- modify the first field based on the pattern relevance;
- detect a contextual cue in the first field;
- retrieve contextual information for a value in the first field;
- add the contextual information to the dataset;
- identify a numerical scheme for the first field;
- and
- parse the first field into a number according to the numerical scheme.
Dependencies are inferred from the claim text.