Automated data extraction and adaptation
AssigneeAmerican Express Travel Related Services Company, Inc.
InventorsRares Ioan Almasan, Rebecca L. Henry, Rahul Menon
Life of this patent
Estimating the term…
Abstract
Systems and methods for automated data extraction and adaptation are disclosed. The system may receive a data input from an external source using various different input channels. The system may determine a data quality of the data input by comparing data fields of the data input to known metadata in the system. The system may reformat the data input based on the comparison to a format consumable by downstream applications and services. The system may apply various machine learning operations on the data input including a descriptive analytics analysis, a predictive learning analysis, and/or a prescriptive intelligence analysis.
Claims20
A method, comprising:
- receiving, by a computer-based system, a data input from an input channel, wherein the data input comprises at least one data field that defines a format of the data input;
- retrieving, by a computer-based system, one or more data structures from a database in electronic communication with the computer-based system, wherein each of the one or more data structures comprises a request type, a format, an input channel, and acknowledgement requirements;
- comparing, by the computer-based system, the at least one data field of the data input to the format of each of the one or more data structures to determine whether the format of the data input corresponds to the format of a data structure;
- when the format of the data input corresponds to the format of a data structure, performing operations comprising:
- reformatting, by the computer-based system, the at least one data field of the data input such that the data input is in the format defined by the format of the data structure that is consumed by downstream platforms, wherein the reformatting comprises encryption, aggregation, and augmentation;
- and
- storing, by the computer-based system, the reformatted data input at the database;
- when the format of the data input is determined to be unknown, performing operations comprising:
- predicting, by the computer-based system, a format associated with the data input, wherein the predicting is performed by one or more machine learning models, wherein the one or more machine learning models have been trained on the one or more data structures and reformatted data stored at the database;
- reformatting, by the computer-based system, the data input based on the format generated by the one or more machine learning models;
- updating, by the computer-based system, the database by storing the reformatted data input at the database;
- training, by the computer-based system, each of the one or more machine learning models with the reformatted data input stored at the updated database;
- and
- analyzing, by the computer-based system using machine learning, the at least one data field of the data input by one or more of the machine learning models to generate a machine learning analysis output that provides an insight into the format of the data input,
- wherein the standardized format organizes the at least one data field of the data input in a predefined order, and
- wherein the insight into the format of the data input comprises at least one of a trend, a causal event, and a correlated event of the data input.
A system comprising:
- a processor;
- and
- a tangible, non-transitory memory configured to communicate with the processor, the tangible, non-transitory memory having instructions stored thereon that, in response to execution by the processor, cause the processor to perform operations comprising:
- receiving, by the processor, a data input from an input channel, wherein the data input comprises at least one data field that defines a format of the data input;
- retrieving, by the processor, one or more data structures from a database in electronic communication with the processor, wherein each of the one or more data structures comprises a request type, a format, an input channel, and acknowledgement requirements;
- comparing, by the processor, the at least one data field of the data input to the format of each of the one or more data structures to determine whether the format of the data input corresponds to the format of a data structure;
- when the format of the data input corresponds to the format of a data structure, performing operations comprising:
- reformatting, by the processor, the at least one data field of the data input such that the data input is in the format defined by the format of the data structure that is consumed by downstream platforms, wherein the reformatting comprises encryption, aggregation, and augmentation and;
- storing, by the processor, the reformatted data at the database
- when the format of the data input is determined to be unknown, performing operations comprising:
- predicting, by the processor, a format associated with the data input, wherein the predicting is performed by one or more machine learning models, wherein the one or more machine learning models have been trained on the one or more data structures and reformatted data stored at the database;
- reformatting, by the processor, the data input based on the format generated by the one or more machine learning models;
- updating, by the processor, the database by storing the reformatted data input at the database;
- training, by the processor, each of the one or more machine learning models with the reformatted data input stored at the updated database;
- and
- analyzing, by the processor using machine learning, the at least one data field of the data input by one or more of the machine learning models to generate a machine learning analysis output that provides an insight into the format of the data input,
- wherein the standardized format organizes the at least one data field of the data input in a predefined order, and
- wherein the insight into the format of the data input comprises at least one of a trend, a causal event, and a correlated event of the data input.
An article of manufacture including a non-transitory, tangible computer readable storage medium having instructions stored thereon that, in response to execution by a computer-based system, cause the computer-based system to perform operations comprising:
- receiving, by the computer-based system, a data input from an input channel, wherein the data input comprises at least one data field that defines a format of the data input;
- retrieving, by the computer-based system, one or more data structures from a database in electronic communication with the computer-based system, wherein each of the one or more data structures comprises a request type, a format, an input channel, and acknowledgement requirements;
- comparing, by the computer-based system, the at least one data field of the data input to the format of each of the one or more data structures to determine whether the format of the data input corresponds to the format of a data structure;
- when the format of the data input corresponds to the format of a data structure, performing operations comprising:
- reformatting, by the computer-based system, the at least one data field of the data input such that the data input is in the format defined by the format of the data structure that is consumed by downstream platforms, wherein the reformatting comprises encryption, aggregation, and augmentation;
- and
- storing, by the computer-based system, the reformatted data at the database;
- when the format of the data input is determined to be unknown, performing operations comprising:
- predicting, by the computer-based system, a format associated with the data input, wherein the predicting is performed by one or more machine learning models, wherein the one or more machine learning models have been trained on the one or more data structures and reformatted data stored at the database;
- reformatting, by the computer-based system, the data input based on the format generated by the one or more machine learning models into the standardized format;
- updating, by the computer-based system, the database by storing the reformatted data input at the database;
- training, by the computer-based system, each of the one or more machine learning models with the reformatted data input stored at the updated database;
- and
- analyzing, by the computer-based system using machine learning, the at least one data field of the data input by one or more of the machine learning models to generate a machine learning analysis output that provides an insight into the format of the data input,
- wherein the standardized format organizes the at least one data field of the data input in a predefined order,
- wherein the insight into the format of the data input comprises at least one of a trend, a causal event, and a correlated event of the data input, and
- wherein to generate the machine learning analysis output further comprises:
- generating, by the computer-based system, a descriptive analytics output in response to a first data field of the data input corresponding to the format of a data structure;
- assessing, by the computer-based system, the data input using a predictive learning analysis to determine a second data field of the data input that is not included in the format of a data structure;
- generating, by the computer-based system, a machine learning analysis output using a prescriptive intelligence analysis based on the descriptive analytics output and the predictive learning analysis;
- and
- enhancing, by the computer-based system, the input channel based on the machine learning analysis output.
Dependencies are inferred from the claim text.