US 12,608,394 B2Granted

Comprehensive data integration implementing seamless extraction, loading and transformation of data

AssigneeBank of America Corporation

InventorsSaket Vilas Samel, Rajneesh Acharya, Marlon R. Aguirre, Navdeep Gadhoke, Bindya Jain, Naveen Kumar Jayaraman, Tushar Kale, Ramesh Krishnan, Manikandan Madhu, Kiranswamy Malemath, Vinay Manhass, Raghavender Singh Panthulu, Mandar Parab, Karthik Rajapalani, Siddharthan Sethuraj

Life of this patent

Estimating the term…

patentgrep

Loading patent…

Abstract

A comprehensive/unified platform for data integration/processing including building/generating and managing data applications is disclosed. The data integration platform takes a modular approach such that new functionality can easily be added to the platform as future needs dictate. In this regard, the data integration platform includes modules for data ingestion, data transformation, data loading/storage, data analytics and the like that provide end-to-end functionality. The data integration platform provides for unified data operations capabilities, including data governance, data quality control, data control and data lineage. The data integration platform may leverage the use AI including ML, such as generative AI to facilitate the capturing of metadata, enhancing/improving captured data, and assisting in data analytics.

Claims19

Dependent claims

A system for comprehensive data integration, the system comprising:

a computing platform including a memory and one or more computing processor devices in communication with the memory, wherein the memory stores a data integration platform that is executable by at least one of the one or more computing processor devices and includes:
a user interface module configured to receive user inputs from a user that define metadata for processing a set of data including (i) data ingestion, (ii) data transformation, (iii) data storage and (iv) data analyzation;
a data ingestion module configured to manage intake of the set of data in accordance with the metadata;
a data transformation module includes:
a rules engine defining a plurality of rules, wherein the data transformation module is configured to apply (i) logic and (ii) one or more of the plurality of rules applicable to the metadata to perform a process that transforms the set of data to a data processing output;
a Machine Learning (ML)-based (i) queuing model configured to generate simulated data associated with a wait time for processing outstanding sets of data awaiting transformation based on rate of incoming data transformation requests, rate of request transformation, and number of data transformation instances and (ii) predictive model configured to use, at least, the simulated data to predict a processing completion time for (a) sets of data currently being transformed and (b) sets of data awaiting transformation;
automatically adjust processing capabilities based on at least the predicted processing completion time, wherein the processing capabilities comprise at least increasing a server capacity;
a data storage module configured to organize and store the set of data and the data processing output in accordance with the metadata;
and
a data analytics module comprising Artificial Intelligence (AI) including one or more Machine Learning (ML) models configured to allow the user to perform data analysis on, at least, the data processing output,
wherein the data integration platform includes data operations capabilities including (i) data governance, (ii) data quality controls, (iii) data control, and (iv) data lineage configured to be implemented throughout the data integration platform.

8 more elements

A computer-implemented method for comprehensive data integration, the computer-implemented method executed by one or more computing processor devices within a single data integration platform and comprising:

receiving user inputs from a user that define metadata for processing a set of data including (i) data ingestion, (ii) data transformation, (iii) data storage and (iv) data analyzation;
managing intake of the set of data in accordance with the metadata;
applying (i) logic and (ii) one or more rules applicable to the metadata to perform a process that transforms the set of data to a data processing output;
organizing and storing the set of data and the data processing output in accordance with the metadata;
implementing Artificial Intelligence including Machine Learning (ML) techniques to perform data analysis on, at least, the data processing output, wherein implementing further comprises:
generating, using a queuing model, simulated data associated with a wait time for processing outstanding sets of data awaiting transformation based on rate of incoming data transformation requests, rate of request transformation, and number of data transformation instances;
and
predicting, using a predictive model, a processing completion time for (a) sets of data currently being transformed and (b) sets of data awaiting transformation using the simulated data;
automatically adjusting processing capabilities based on at least the predicted processing completion time, wherein the processing capabilities comprise at least increasing a server capacity;
and
implementing data operations capabilities including (i) data governance, (ii) data quality controls, (iii) data control, and (iv) data lineage in conjunction with the steps of the method.

8 more elements

A computer program product comprising:

a non-transitory computer-readable medium comprising sets of codes for causing one or more computing devices to:
receive user inputs from a user that define metadata for processing a set of data including (i) data ingestion, (ii) data transformation, (iii) data storage and (iv) data analyzation;
manage intake of the set of data in accordance with the metadata;
apply (i) logic and (ii) one or more rules applicable to the metadata to perform a process that transforms the set of data to a data processing output;
organize and store the set of data and the data processing output in accordance with the metadata;
implement Artificial Intelligence including Machine Learning (ML) techniques to perform data analysis on, at least, the data processing output, wherein implementing further comprises:
generating, using a queuing model, simulated data associated with a wait time for processing outstanding sets of data awaiting transformation based on rate of incoming data transformation requests, rate of request transformation, and number of data transformation instances;
and
predicting, using a predictive model, a processing completion time for (a) sets of data currently being transformed and (b) sets of data awaiting transformation using the simulated data;
automatically adjust processing capabilities based on at least the predicted processing completion time, wherein the processing capabilities comprise at least increasing a server capacity;
and
implement data operations capabilities including (i) data governance, (ii) data quality controls, (iii) data control, (and iv) data lineage in conjunction with any of the sets of codes.

9 more elements

Dependencies are inferred from the claim text.