# US 2023/0066110 A1 · Creating virtualized data assets using existing definitions of etl/elt jobs

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0066110 A1
- Document ID: US20230066110A1
- Status: Published application
- Kind: A1
- Application number: 17/462,349
- Filed: 2021-08-31
- Published: 2023-03-02
- Earliest claimed: 2021-08-31
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Michal Bodziony; Bartosz Tomasik; Wojciech Mis; Marcin Filip
- Web page: https://patentgrep.com/patent/US20230066110A1

## CPC

- G06F 16/24535: {of sub-queries or views}
- G06F 16/24565: {Triggers; Constraints}
- G06F 16/254: {Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses}
- G06F 16/256: {in federated or virtual databases}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A computer-implemented method, system and computer program product for creating virtualized data assets. A copy of an extract, transform, load (ETL) or an extract, load, transform (ELT) job definition is created. Furthermore, a table function is created in which the table function is configured to trigger the execution of a copy of the ETL/ELT job definition, to read the resulting data from executing the copy of the ETL/ELT job definition and to return the resulting data (data sets), such as in a queue in a federation server. After receiving a query from a user based on a virtual view in which the query includes the ETL/ELT job definition, the table function is executed which triggers execution of the copy of the ETL/ELT job definition. Data sets are then produced by executing the copied ETL/ELT job definition, which is read and returned to the user in the virtual view.

## Claims

### Claim 1

A computer-implemented method for creating virtualized data assets, the method comprising:
receiving a set of extract, transform, load (ETL) or extract, load, transform (ELT) job definitions;
translating said set of ETL or ELT job definitions to statements in a domain-specific language;
creating virtualized data assets using said translated statements; and
representing a federated view for execution of said translated statements that includes said created virtualized data assets.

### Claim 2 (depends on 1)

The method as recited in claim 1 further comprising:
building a first catalog to store said created virtualized data assets with corresponding ETL or ELT job definitions.

### Claim 3 (depends on 2)

The method as recited in claim 2 further comprising:
creating a second set of ETL or ELT job definitions to use associated translated statements to be executed against said virtualized data assets of said first catalog.

### Claim 4 (depends on 3)

The method as recited in claim 3 further comprising:
building a second catalog defining how data is extracted, transformed and loaded or how data is extracted, loaded and transformed (ELT) based on said set of ETL or ELT job definitions, respectively, wherein said second catalog is modified by replacing said set of ETL or ELT job definitions with said created second set of ETL or ELT job definitions.

### Claim 5 (depends on 1)

The method as recited in claim 1, wherein said domain-specific language comprises a structured query language.

### Claim 6 (depends on 1)

The method as recited in claim 1, wherein said translation of said set of ETL or ELT job definitions to statements in said domain-specific language is accomplished via a table of ETL or ELT stages with corresponding statements in said domain-specific language.

### Claim 7 (depends on 1)

The method as recited in claim 1 further comprising:
storing said translated statements in a third catalog.

### Claim 8

A computer-implemented method for creating virtualized data assets, the method comprising:
creating a copy of an extract, transform, load (ETL) or an extract, load, transform (ELT) job definition;
creating a table function for triggering execution of said copy of said ETL or ELT job definition and returning data sets;
receiving a query from a user based on a virtual view, wherein said query comprises said ETL or ELT job definition;
executing said table function which triggers execution of said copy of said ETL or ELT job definition in response to receiving said query;
producing said data sets in response to execution of said copy of said ETL or ELT job definition; and
reading and returning said data sets in said virtual view.

### Claim 9 (depends on 8)

The method as recited in claim 8 further comprising:
creating a queue to store results from execution of said copy of said ETL or ELT job definition.

### Claim 10 (depends on 9)

The method as recited in claim 9 further comprising:
modifying said copy of said ETL or ELT job definition to store results of said execution of said copy of said ETL or ELT job definition in said queue.

### Claim 11 (depends on 8)

The method as recited in claim 8 further comprising:
creating said table function for reading said data sets from executing said copy of said ETL or ELT job definition.

### Claim 12 (depends on 8)

The method as recited in claim 8, wherein said virtual view comprises a single view of virtualized data obtained from a plurality of data sources, wherein said plurality of data sources comprises data warehouses, data marts and data lakes.

### Claim 13 (depends on 8)

The method as recited in claim 8 further comprising:
receiving a request from said user to create said virtual view for said ETL or ELT job definition.

### Claim 14 (depends on 13)

The method as recited in claim 13 further comprising:
creating said virtual view for said user.

### Claim 15

A computer program product for creating virtualized data assets, the computer program product comprising one or more computer readable storage mediums having program code embodied therewith, the program code comprising programming instructions for:
creating a copy of an extract, transform, load (ETL) or an extract, load, transform (ELT) job definition;
creating a table function for triggering execution of said copy of said ETL or ELT job definition and returning data sets;
receiving a query from a user based on a virtual view, wherein said query comprises said ETL or ELT job definition;
executing said table function which triggers execution of said copy of said ETL or ELT job definition in response to receiving said query;
producing said data sets in response to execution of said copy of said ETL or ELT job definition; and
reading and returning said data sets in said virtual view.

### Claim 16 (depends on 15)

The computer program product as recited in claim 15, wherein the program code further comprises the programming instructions for:
creating a queue to store results from execution of said copy of said ETL or ELT job definition.

### Claim 17 (depends on 16)

The computer program product as recited in claim 16, wherein the program code further comprises the programming instructions for:
modifying said copy of said ETL or ELT job definition to store results of said execution of said copy of said ETL or ELT job definition in said queue.

### Claim 18 (depends on 15)

The computer program product as recited in claim 15, wherein the program code further comprises the programming instructions for:
creating said table function for reading said data sets from executing said copy of said ETL or ELT job definition.

### Claim 19 (depends on 15)

The computer program product as recited in claim 15, wherein said virtual view comprises a single view of virtualized data obtained from a plurality of data sources, wherein said plurality of data sources comprises data warehouses, data marts and data lakes.

### Claim 20 (depends on 15)

The computer program product as recited in claim 15, wherein the program code further comprises the programming instructions for:
receiving a request from said user to create said virtual view for said ETL or ELT job definition.
