# US 12,613,880 B2 · Domain specific language for data pipeline creation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,613,880 B2
- Document ID: US12613880B2
- Status: Granted
- Kind: B2
- Application number: 18/675,915
- Filed: 2024-05-28
- Published: 2025-12-04
- Granted: 2026-04-28
- Earliest claimed: 2024-05-28
- Assignees: Matillion Limited
- Applicants: Matillion Limited
- Inventors: Robert John Appleyard; Mark Bryan Davies; Samuel Thomas Perrin
- Web page: https://patentgrep.com/patent/US12613880B2

## CPC

- G06F 16/254: {Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A domain specific language, the “data pipeline language” (DPL), has been developed for programmatic creation of data pipelines for ETL/ELT with a human-readable format. The DPL syntax is abstracted from the specific technologies used for building a data pipeline, so the DPL that defines various components of a data pipeline (e.g., data sources, data warehouses, and/or cloud-based processing components) is independent of the specific vendors used to provide these components. A user that defines a data pipeline in the DPL specifies any external vendors that will be leveraged for execution of the data pipeline in a configuration file. During execution of a data pipeline defined in with the DPL, a pipeline execution platform maps data pipeline components identified from the DPL file to the respective external vendor(s) indicated in the configuration and “translates” the DPL components into their corresponding commands that are sent to the external vendor(s).

## Claims

### Claim 1

A method comprising:
obtaining a data pipeline file and a configuration file, wherein the data pipeline file comprises a definition of a data pipeline and indicates a plurality of components,
wherein the data pipeline is an extract, transform, load (ETL) or an extract, load, transform (ELT) data pipeline,
wherein the configuration file indicates mappings between types of components included in the data pipeline and one or more external vendors to which components of the indicated types correspond,
wherein the definition of the data pipeline comprises human-readable syntax abstracted from the one or more external vendors; and
executing the data pipeline, wherein executing the data pipeline comprises, for at least a first component of the plurality of components of the data pipeline,
determining that the first component corresponds to at least one of a data source, a data warehouse, and a processing component of a first external vendor of the one or more external vendors based on the configuration file and a type of the first component indicated in the data pipeline file;
translating the first component to a first command that corresponds to the first external vendor, wherein translating the first component to the first command comprises mapping the first component to the first command based on the type of the first component and the mappings between types of components and the one or more external vendors indicated in the configuration file; and
communicating the first command to the first external vendor to perform one or more actions corresponding to execution of the first component of the data pipeline.

### Claim 2 (depends on 1)

The method of claim 1, wherein the data pipeline file comprises structured data, wherein the structured data comprises a plurality of attributes and a corresponding plurality of values, wherein each of the plurality of components comprises a subset of the plurality of attributes and a corresponding subset of the plurality of values.

### Claim 3 (depends on 1)

The method of claim 1, wherein communicating the first command to the first external vendor comprises communicating the first command to the first external vendor via an application programming interface (API) of the first external vendor.

### Claim 4 (depends on 1)

The method of claim 1, further comprising constructing the first command, wherein constructing the first command comprises determining one or more parameter values indicated in program code of the first component and incorporating the one or more parameter values into the first command.

### Claim 5 (depends on 1)

The method of claim 1 further comprising, based on determining that execution of the first command was successful, executing one or more subsequent components of the data pipeline indicated by one or more attributes of the first component, wherein determining that execution of the first command was successful is based on obtaining a response to the first command from the first external vendor.

### Claim 6 (depends on 1)

The method of claim 1, further comprising determining that execution of the data pipeline has been triggered based on detecting selection of an element of a graphical user interface (GUI), wherein the GUI comprises a depiction of the data pipeline.

### Claim 7

One or more non-transitory machine-readable media having program code stored thereon, the program code comprising instructions to:
obtain a data pipeline file and a configuration file, wherein the data pipeline file comprises a definition of a data pipeline and indicates a plurality of components,
wherein the data pipeline is an extract, transform, load (ETL) or an extract, load, transform (ELT) data pipeline,
wherein the configuration file indicates mappings between types of components included in the data pipeline and one or more external vendors to which components of the indicated types correspond,
wherein the definition of the data pipeline comprises human-readable syntax abstracted from the one or more external vendors; and
execute the data pipeline, wherein the instructions to execute the data pipeline comprise instructions to, for at least a first component of the plurality of components of the data pipeline,
determine that the first component corresponds to at least one of a data source, a data warehouse, and a processing component of a first external vendor of the one or more external vendors based on the configuration file and a type of the first component indicated in the data pipeline file;
translate the first component to a first command that corresponds to the first external vendor, wherein the instructions to translate the first component to the first command comprise instructions to map the first component to the first command based on the type of the first component and the mappings between types of components and the one or more external vendors indicated in the configuration file; and
communicate the first command to the first external vendor to perform one or more actions corresponding to execution of the first component of the data pipeline.

### Claim 8 (depends on 7)

The non-transitory machine-readable media of claim 7, wherein the instructions to communicate the first command to the first external vendor comprise instructions to communicate the first command to the first external vendor via an application programming interface (API) of the first external vendor.

### Claim 9 (depends on 7)

The non-transitory machine-readable media of claim 7, wherein the program code further comprises instructions to construct the first command, wherein the instructions to construct the first command comprise instructions to determine one or more parameter values indicated in one or one attributes of the first component and incorporate the one or more parameter values into the first command.

### Claim 10 (depends on 7)

The non-transitory machine-readable media of claim 7, wherein the program code further comprises instructions to, based on a determination that execution of the first command was successful, execute one or more subsequent components of the data pipeline, wherein the instructions to determine that execution of the first command was successful comprise instructions to determine that execution of the first command was successful based on obtaining a response to the first command from the first external vendor.

### Claim 11 (depends on 7)

The non-transitory machine-readable media of claim 7, wherein the data pipeline file comprises structured data, wherein the structured data comprises a plurality of attributes and a corresponding plurality of values, wherein each of the plurality of components comprises a subset of the plurality of attributes and a corresponding subset of the plurality of values.

### Claim 12

An apparatus comprising:
a processor; and
a machine-readable medium having instructions stored thereon that are executable by the processor to cause the apparatus to,
obtain a data pipeline file and a configuration file, wherein the data pipeline file comprises a definition of a data pipeline and indicates a plurality of components, wherein the data pipeline is an extract, transform, load (ETL) or an extract, load, transform (ELT) data pipeline,
wherein the configuration file indicates mappings between types of components included in the data pipeline and one or more external vendors to which components of the indicated types correspond,
wherein the definition of the data pipeline comprises human-readable syntax abstracted from the one or more external vendors; and
execute the data pipeline, wherein the instructions to execute the data pipeline comprise instructions to, for at least a first component of the plurality of components of the data pipeline,
determine that the first component corresponds to at least one of a data source, a data warehouse, and a processing component of a first external vendor of the one or more external vendors based on the configuration file and a type of the first component indicated in the data pipeline file; and
translate the first component to a first command that corresponds to the first external vendor, wherein the instructions executable by the processor to cause the apparatus to translate the first component to the first command comprise instructions executable by the processor to cause the apparatus to map the first component to the first command based on the type of the first component and the mappings between types of components and the one or more external vendors indicated in the configuration file; and
communicate the first command to the first external vendor to perform one or more actions corresponding to execution of the first component of the data pipeline.

### Claim 13 (depends on 12)

The apparatus of claim 12, wherein the instructions executable by the processor to cause the apparatus to communicate the first command to the first external vendor comprise instructions executable by the processor to cause the apparatus to communicate the first command to the first external vendor via an application programming interface (API) of the first external vendor.

### Claim 14 (depends on 12)

The apparatus of claim 12, further comprising instructions executable by the processor to cause the apparatus to construct the first command, wherein the executable by the processor to cause the apparatus to construct the first command comprise instructions executable by the processor to cause the apparatus to determine one or more parameter values indicated by one or more attributes of the first component and incorporate the one or more parameter values into the first command.

### Claim 15 (depends on 12)

The apparatus of claim 12, further comprising instructions executable by the processor to cause the apparatus to, based on obtaining a response to the first command from the first external vendor indicating that execution of the first command was successful, execute one or more subsequent components of the data pipeline.

### Claim 16 (depends on 12)

The apparatus of claim 12, wherein the data pipeline file comprises structured data, wherein the structured data comprises a plurality of attributes and a corresponding plurality of values, wherein each of the plurality of components comprises a subset of the plurality of attributes and a corresponding subset of the plurality of values.

### Claim 17 (depends on 12)

The apparatus of claim 12, further comprising instructions executable by the processor to cause the apparatus to determine that execution of the data pipeline has been triggered based on detection of selection of an element of a graphical user interface (GUI), wherein the GUI comprises a depiction of the data pipeline.

### Claim 18 (depends on 1)

The method of claim 1, wherein determining that the first component corresponds to at least one of a data source, a data warehouse, and a processing component of the first external vendor based on the configuration file and the type of the first component comprises,
determining the type of the first component indicated in the data pipeline file; and
determining that the mappings indicated in the configuration file comprise a first mapping between the type of the first component and the first external vendor.

### Claim 19 (depends on 7)

The non-transitory machine-readable media of claim 7, wherein the instructions to determine that the first component corresponds to at least one of a data source, a data warehouse, and a processing component of the first external vendor based on the configuration file and the type of the first component indicated in the data pipeline file comprise instructions to,
determining the type of the first component indicated in the data pipeline file; and
determining that the mappings indicated in the configuration file comprise a first mapping between the type of the first component and the first external vendor.

### Claim 20 (depends on 12)

The apparatus of claim 12, wherein the instructions executable by the processor to cause the apparatus to determine that the first component corresponds to at least one of a data source, a data warehouse, and a processing component of the first external vendor based on the configuration file and the type of the first component indicated in the data pipeline file comprise instructions executable by the processor to cause the apparatus to,
determine the type of the first component indicated in the data pipeline file; and
determine that the mappings indicated in the configuration file comprise a first mapping between the type of the first component and the first external vendor.
