US 12,182,280 B2Granted

Data transformation engine

AssigneeData Solutions, Inc.

InventorsLoren John Ball, Eduardo Bayudan

Life of this patent

Estimating the term…

Loading patent…

Abstract

The data transformation method operates discrete input data sources DF-in each having df-in data fields carrying to-be-converted data (tbc-data). Initial steps identify tbc-data field and characteristics; output discrete output (df-out) with predefined df-out data fields for converted data (conv-data); and maps or look-up tables for relationships between df-in data fields and df-out data fields. Process activates: SFTP App in the presence df-in and transfers df-in to unstructured data blob; and App copying the df-in from blob to file share archive; first orchestrating sequence App to validate df-in data based upon conversion rules. If INVALID, App generates error entry. If VALID, App converts tbc-data into conv-data with conversion rules and mapping orchestrating App populates df-out data fields with conv-data resulting converted df-out in the blob. A second orchestrating App transfers converted df-out to destination store. Copying App copies converted df-out from blob to archive and deletes converted df-out from blob.

Claims8

Dependent claims

A data transformation method comprising:

operating on a plurality of input data sources (d-s) comprising a plurality of discrete data files and discrete databases (df-in), each df-in having a plurality of df-in data fields each with to-be-converted data (tbc-data);
initially defining data field conversion rules including tbc-data field length, tbc-data format, tbc-data data value limits, tbc-data null values, and a tbc-data present condition;
initially defining a plurality of output discrete data files and discrete databases (df-out), each df-out having a plurality of df-out data fields for converted data (conv-data);
initially defining one or more data field maps or look-up tables which define relationships between df-in data fields and df-out data fields;
activating an application (App) for secure file transfer function (sft-App) in the presence of one of the plurality of discrete data files or discrete databases (cl-source) which transfers the cl-source into a large data store for unstructured data (blob);
activating a copy App copying the cl-source from the blob to a file share archive resulting in an archive cl-source;
activating a first orchestrating performance sequence App on cl-source validating the tbc-data in the plurality of df-in data fields based upon the conversion rules;
generating corresponding error log entry when tbc-data in the plurality of df-in data fields is invalid;
converting the tbc-data into conv-data with the conversion rules and activating a mapping orchestrating performance sequence App which populates the plurality of df-out data fields with corresponding conv-data resulting in a converted df-out in the blob when tbc-data in the plurality of df-in data fields is valid;
activating a second orchestrating performance sequence App transferring converted df-out to a df-out destination data store via the sft-App;
and
copying converted df-out from the blob to the file share archive resulting in an archive df-out and then deleting converted df-out from the blob.

9 more elements

A data transformation method comprising:

operating on a plurality of input data sources (d-s) comprising a plurality of discrete data files and discrete databases (df-in), each df-in having a plurality of df-in data fields each with to-be-converted data (tbc-data);
initially defining data field conversion rules including tbc-data field length, tbc-data format, and tbc-data data value limits;
initially defining a plurality of output discrete data files and discrete databases (df-out), each df-out having a plurality of df-out data fields for converted data (conv-data);
initially defining one or more data field maps or look-up tables which either (i) define relationships between df-in data fields and df-out data fields and (ii) algorithmic operations on df-in data fields to generate conv-data for the corresponding df-out data field;
activating an application (App) for secure file transfer function (sft-App) in the presence of the one of the plurality of discrete data files or discrete databases (cl-source) which transfers the cl-source into a large data store for unstructured data (blob);
activating a copy App copying the cl-source from the blob to a file share archive resulting in an archive cl-source;
activating a first orchestrating performance sequence App on cl-source validating the tbc-data based upon the conversion rules;
generating corresponding error log entry when tbc-data in a corresponding df-in data field is invalid;
converting the tbc-data into conv-data with the conversion rules and activating a mapping orchestrating performance sequence App which populates the plurality of df-out data fields with corresponding conv-data resulting in a converted df-out in the blob when tbc-data is valid;
activating a second orchestrating performance sequence App transferring converted df-out to a destination data store;
and
copying converted df-out from the blob to the file share archive resulting in an archive df-out and then deleting converted df-out from the blob.

9 more elements

A data transformation method comprising:

operating on a plurality of input data sources (d-s) comprising a plurality of discrete data files and discrete databases (df-in), each df-in having a plurality of df-in data fields each with to-be-converted data (tbc-data);
initially defining a plurality of output discrete data files and discrete databases (df-out), each df-out having a plurality of df-out data fields for converted data (conv-data);
initially defining data field validation rules including tbc-data field length, tbc-data format, and tbc-data data value limits;
initially defining data field ingestion rules including ingestion-data (i-d) field length, i-d data format, and tbc-data data value limits;
the data field ingestion rules further including one or more data field maps or look-up tables which either (i) define relationships between df-in data fields and df-out data fields and (ii) algorithmic operations on df-in data fields in operation, the data field ingestion rules generating conv-data for corresponding df-out data fields;
activating an application (App) for secure file transfer function (sft-App) in the presence of the one of the plurality of discrete data files or discrete databases (cl-source) which transfers the cl-source into a large data store for unstructured data (blob);
activating a copy App copying the cl-source from the blob to a file share archive resulting in an archive cl-source;
activating a first orchestrating performance sequence App on cl-source validating the tbc-data based upon the conversion rules;
generating corresponding error log entry when tbc-data in a corresponding df-in data field is invalid;
converting the tbc-data into conv-data with the conversion rules and activating a mapping orchestrating performance sequence App which populates the plurality of df-out data fields with corresponding conv-data resulting in a converted df-out in the blob when tbc-data is valid;
activating a second orchestrating performance sequence App applying the ingestion rules to generate converted df-out to a destination data store;
and
copying converted df-out from the blob to the file share archive resulting in an archive df-out and then deleting converted df-out from the blob.

10 more elements

Dependencies are inferred from the claim text.