Date and time feature identification
AssigneeInternational Business Machines Corporation
InventorsSatoshi Masuda, Takaaki Tateishi, Toshihiro Takahashi
Life of this patent
Estimating the term…
Abstract
Methods and systems for text processing include building a knowledge base using column names and associated functions from a code base. Classifiers are trained using the knowledge base and are cross-validated to determine accuracy scores. Text is processed using a selected classifier having a highest accuracy score from the classifiers to determine date/time features.
Claims20
A computer-implemented method for text processing, comprising:
- building a knowledge base of date/time features using column names and associated functions from parsed computer source codes extracted from a code base;
- training a plurality of classifiers using the knowledge base;
- cross-validating the plurality of classifiers to determine accuracy scores by comparing labels generated in output layers of the plurality of classifiers using a validation dataset to the knowledge base of date/time features;
- processing text using a selected classifier having a highest accuracy score from the plurality of classifiers to determine date/time features from the text knowledge base of date/time features;
- generating new column information for database tables by merging column data with the date/time features determined from the text knowledge base of date/time features;
- and
- training machine learning models with the new column information for the database tables to increase accuracy of prediction of the machine learning models.
A computer program product for text processing, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions being executable by a hardware processor to cause the hardware processor to:
- build a knowledge base of date/time features using column names and associated functions from parsed computer source codes extracted from a code base;
- train a plurality of classifiers using the knowledge base;
- cross-validate the plurality of classifiers to determine accuracy scores by comparing labels generated in output layers of the plurality of classifiers using a validation dataset to the knowledge base of date/time features;
- process text using a selected classifier having a highest accuracy score from the plurality of classifiers to determine date/time features from the knowledge base of date/time features;
- generate new column information for database tables with the date/time features determined from the text knowledge base of date/time features;
- and
- train machine learning models with the new column information for the database tables to increase accuracy of prediction of machine learning models.
A system for text processing, comprising:
- a hardware processor;
- and
- a memory that includes a computer program, which, when executed by the hardware processor, causes the hardware processor to:
- build a knowledge base of date/time features using column names and associated functions from parsed computer source codes extracted from a code base;
- train a plurality of classifiers using the knowledge base;
- cross-validate the plurality of classifiers to determine accuracy scores by comparing labels generated in output layers of the plurality of classifiers using a validation dataset to the knowledge base of date/time features;
- process text using a selected classifier having a highest accuracy score from the plurality of classifiers to determine date/time features from the text knowledge base of date/time features;
- generate new column information for database tables with the date/time features determined from the text knowledge base of date/time features;
- and
- train machine learning models with the new column information for the database tables to increase accuracy of prediction of machine learning models.
Dependencies are inferred from the claim text.