US 12,681,950 B2Granted

Systems, methods, and apparatuses for combining, interpreting, and distributing non-uniform datasets into structured datasets

AssigneeBank of America Corporation

InventorsNatalia Bakholdina, Andrew Stellman, Alexander James Diestler, Joseph Eric Pringle, Mayer Crystal

Life of this patent

Estimating the term…

Loading patent…

Abstract

Systems, computer program products, and methods are described herein for a system for combining, interpreting, and distributing non-uniform datasets into structured datasets, the system comprising: a memory device with computer-readable program code stored thereon, a communication device, and a processing device operatively coupled to the memory device and the communication device. The processing device identifies a configuration file with location information for at least one dataset or script file, retrieves the dataset, builds a tree of execution comprising at least one node wherein each node comprises data, executes the script file on two or more datasets in parallel on a computational cluster comprising at least one computing component, wherein the at least one computing component comprises an at least one node and generates an output comprising as a structured dataset, wherein the structured dataset comprises a standardized format associated with the at least one instruction of the script file.

Claims19

Dependent claims

A system for a distributed execution engine for combining, interpreting, and distributing non-uniform datasets into structured datasets, the system comprising:

a memory device with computer-readable program code stored thereon;
a communication device;
a processing device operatively coupled to the memory device and the communication device, wherein the processing device is configured to execute the computer-readable program code to:
identify at least one configuration file, the at least one configuration file comprising at least one location associated with one or more datasets or at least one script file comprising at least one instruction;
identify the at least one script file based on the at least one location associated with the at least one configuration file;
retrieve the one or more datasets based on the at least one location associated with the at least one configuration file, wherein the one or more datasets comprise one or more flexible schemas organized based on data available in the one or more datasets;
identify, by an execution engine, at least one instruction from the at least one script file, wherein the at least one instruction is written in a data centric language;
build, by the execution engine and using the at least one instruction, a tree of execution based on the one or more datasets and the data available in the one or more datasets, wherein the tree of execution comprises a plurality of nodes, and wherein the plurality of nodes comprises the at least one instruction corresponding to the data available and the one or more flexible schemas, and wherein each node comprises a key value pair corresponding to the data available in each node;
execute the at least one instruction at each node based on the key value pair on the one or more datasets in the tree of execution, wherein the key value pair indicates at least one present attribute and at least one present value from the data available at each node, wherein the execution is performed in parallel for at least two or more nodes in the tree of execution and using at least one computational cluster comprising at least one computing component at each node and wherein the two or more nodes work together to process data from the one or more datasets using sub-tasks from the at least one script file;
and generate an output comprising a structured dataset from the one or more datasets and based on the execution of the at least one instruction using the tree of execution, wherein the structured dataset comprises a standardized format of the one or more datasets based on the key value pair for each dataset of the one or more datasets.

7 more elements

A computer program product for a distributed execution engine for combing, interpreting, and distributing non-uniform datasets into structured datasets, the computer program product comprising a non-transitory computer-readable medium having computer-readable program code portions embodied therein, the computer-readable program code portions comprising:

an executable portion configured for identifying at least one configuration file, the at least one configuration file comprising at least one location associated with one or more datasets or at least one script file comprising at least one instruction;
an executable portion configured for identifying the at least one script file based on the at least one location associated with the at least one configuration file;
an executable portion configured for retrieving the one or more datasets based on the at least one location associated with the at least one configuration file, wherein the one or more datasets comprise one or more flexible schemas organized based on data available in the one or more datasets;
an executable portion configured for identifying, by an execution engine, at least one instruction from the at least one script file, wherein the at least one instruction is written in a data centric language;
an executable portion configured for building, by the execution engine and using the at least one instruction, a tree of execution based on the one or more datasets and the data available in the one or more datasets, wherein the tree of execution comprises a plurality of nodes, and wherein the plurality of nodes comprises the at least one instruction corresponding to the data available and the one or more flexible schemas, and wherein each node comprises a key value pair corresponding to the data available in each node;
an executable portion configured for executing the at least one instruction at each node based on the key value pair on the one or more datasets in the tree of execution, wherein the key value pair indicates at least one present attribute and at least one present value from the data available at each node, wherein the execution is performed in parallel for at least two or more nodes in the tree of execution and using at least one computational cluster comprising at least one computing component at each node and wherein the two or more nodes work together to process data from the one or more datasets using sub-tasks from the at least one script file;
and an executable portion configured for generating an output comprising a structured dataset from the one or more datasets and based on the execution of the at least one instruction using the tree of execution, wherein the structured dataset comprises a standardized format of the one or more datasets based on the key value pair for each dataset of the one or more datasets associated with the at least one instruction of the script file.

4 more elements

A computer-implemented method for a distributed execution engine for combining, interpreting, and distributing non-uniform datasets into structured datasets, the computer-implemented method comprising:

identifying at least one configuration file, the at least one configuration file comprising at least one location associated with one or more datasets or at least one script file comprising at least one instruction;
identifying the at least one script file based on the at least one location associated with the at least one configuration file;
retrieving the one or more datasets based on the at least one location associated with the at least one configuration file, wherein the one or more datasets comprise one or more flexible schemas organized based on data available in the one or more datasets;
identifying, by an execution engine, at least one instruction from the at least one script file, wherein the at least one instruction is written in a data centric language;
building, by the execution engine and using the at least one instruction, a tree of execution based on the one or more datasets and the data available in the one or more datasets, wherein the tree of execution comprises a plurality of nodes, and wherein the plurality of nodes comprises the at least one instruction corresponding to the data available and the one or more flexible schemas, and wherein each node comprises a key value pair corresponding to the data available in each node;
executing the at least one instruction at each node based on the key value pair on the one or more datasets in the tree of execution, wherein the key value pair indicates at least one present attribute and at least one present value from the data available at each node, wherein the execution is performed in parallel for at least two or more nodes in the tree of execution and using at least one computational cluster comprising at least one computing component at each node and wherein the two or more nodes work together to process data from the one or more datasets using sub-tasks from the at least one script file;
and generating an output comprising a structured dataset from the one or more datasets and based on the execution of the at least one instruction using the tree of execution, wherein the structured dataset comprises a standardized format of the one or more datasets based on the key value pair for each dataset of the one or more datasets.

4 more elements

Dependencies are inferred from the claim text.