# US 2025/0094171 A1 · Systems and methods iterative natural language-based data pipeline generations and prototyping

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0094171 A1
- Document ID: US20250094171A1
- Status: Published application
- Kind: A1
- Application number: 18/598,333
- Filed: 2024-03-07
- Published: 2025-03-20
- Earliest claimed: 2023-09-19
- Assignees: Palantir Technologies Inc.
- Applicants: Palantir Technologies Inc.
- Inventors: Zachary Isaacson; Akshay Agrawal; Rowan Gilmore; Rohith Kasar; Leslie Bresnahan
- Web page: https://patentgrep.com/patent/US20250094171A1

## CPC

- G06F 9/30079: {Pipeline control instructions, e.g. multicycle NOP}
- G06F 16/243: {Natural language query formulation}
- G06F 16/3338: {Query expansion}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

At least some embodiments of the present disclosure are related to methods and systems for evaluating, generating, and/or prototyping data pipelines. In certain embodiments, a system is configured to perform operations include: receiving an input dataset, the input dataset including a data schema; generating a first prompt based on the input dataset and a first prompt structure having one or more text strings and one or more blanks; providing the first prompt to a language model; receiving a use case generated by the language model for the input dataset, the use case including a description of how to use the input dataset; generating a data pipeline based on the use case; and applying the data pipeline to the input dataset to generate an output dataset.

## Claims

### Claim 1

A method for data pipeline evaluations, the method comprising:
receiving an input dataset, the input dataset including a data schema;
generating a first prompt based on the input dataset and a first prompt structure having one or more text strings and one or more blanks;
providing the first prompt to a language model;
receiving a use case generated by the language model for the input dataset, the use case including a description of how to use the input dataset;
generating a data pipeline based on the use case; and
applying the data pipeline to the input dataset to generate an output dataset;
wherein the method is performed using one or more processors.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
generating a second prompt based on a second prompt structure, the second prompt structure associated with the use case and the output dataset;
providing the second prompt to the language model;
receiving an evaluation function generated by the language model; and
determining an evaluation metric using the evaluation function based on the use case and the output dataset.

### Claim 3 (depends on 2)

The method of claim 2, wherein the second prompt structure includes one or more second text strings and one or more second blanks, wherein the generating a second prompt based on a second prompt structure comprises:
determining second text data based on the use case; and
filling in the one or more second blanks using the determined second text data.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
generating a second prompt based on the input dataset, the first prompt structure, the use case, and the evaluation metric;
providing the second prompt to the language model;
receiving a second use case generated by the language model for the input dataset; and
generating a second data pipeline based on the second use case.

### Claim 5 (depends on 2)

The method of claim 2, wherein the use case is a first use case, the output dataset is a first output dataset, and the data pipeline is a first data pipeline, wherein the method further comprises:
receiving a second use case generated by the language model for the input dataset, the second use case including a second description of how to use the input dataset;
generating a second data pipeline based on the second use case; and
applying the second data pipeline to the input dataset to generate a second output dataset.

### Claim 6 (depends on 5)

The method of claim 5, wherein the evaluation function is a first evaluation function and the evaluation metric is a first evaluation metric, wherein the method further comprises:
generating a third prompt based on the second prompt structure, the second use case, and the second output dataset;
providing the second prompt to the language model;
receiving a second evaluation function generated by the language model;
determining a second evaluation metric using the second evaluation function based on the second use case and the second output dataset; and
selecting a use case from the first use case and the second use case based on the first evaluation metric and the second evaluation metric.

### Claim 7 (depends on 1)

The method of claim 1, wherein the language model is a first language model, wherein the generating a data pipeline based on the use case comprises generating the data pipeline using a data pipeline builder including a second language model different from the first language model.

### Claim 8 (depends on 1)

The method of claim 1, wherein the generating a first prompt based on the input dataset and a first prompt structure comprises:
determining text data based on the input dataset; and
filling in the one or more blanks using the determined text data;
wherein the data schema includes one or more data field names and one or more data types;
wherein the determining text data based on the input dataset comprises extracting at least one of the one or more data field names and a corresponding data type from the input dataset.

### Claim 9 (depends on 1)

The method of claim 1, wherein the data pipeline generated based on the use case uses a subset of the input dataset.

### Claim 10 (depends on 1)

The method of claim 1, wherein the receiving an input dataset comprises receiving a selection of the input dataset from one or more input datasets.

### Claim 11 (depends on 1)

The method of claim 1, further comprising:
receiving at least one input selected from a group consisting of: one or more queries, one or more input datasets, and one or more target datasets;
generating a model query based on the at least one input;
generating a query execution plan based at least in part on the model query;
wherein the generating a data pipeline based on the use case comprises:
generating the data pipeline based at least in part on the query execution plan.

### Claim 12 (depends on 11)

The method of claim 11, further comprising:
generating, using one or more computational models, a model result based on the model query;
generating a confidence score associated with the model result;
determining whether the confidence score is higher than a predetermined threshold;
in response to determining that the confidence score is higher than the predetermined threshold, generating the query execution plan;
in response to determining that the confidence score is lower than the predetermined threshold, generating one or more second queries; and
wherein the generating a model query includes generating the model query based on the one or more second queries.

### Claim 13 (depends on 1)

The method of claim 1, wherein the language model includes a large language model.

### Claim 14

A system for data pipeline evaluations, the system comprising:
one or more memories having instructions stored therein; and
one or more processors configured to execute the instructions and perform operations comprising:
receiving an input dataset, the input dataset including a data schema;
generating a first prompt based on the input dataset and a first prompt structure having one or more text strings and one or more blanks;
providing the first prompt to a language model;
receiving a use case generated by the language model for the input dataset, the use case including a description of how to use the input dataset;
generating a data pipeline based on the use case; and
applying the data pipeline to the input dataset to generate an output dataset.

### Claim 15 (depends on 14)

The system of claim 14, wherein the operations further comprise:
generating a second prompt based on a second prompt structure, the second prompt structure associated with the use case and the output dataset;
providing the second prompt to the language model;
receiving an evaluation function generated by the language model; and
determining an evaluation metric using the evaluation function based on the use case and the output dataset.

### Claim 16 (depends on 15)

The system of claim 15, wherein the second prompt structure includes one or more second text strings and one or more second blanks;
wherein, in the operations, the generating a second prompt based on a second prompt structure comprises:
determining second text data based on the use case; and
filling in the one or more second blanks using the determined second text data.

### Claim 17 (depends on 14)

The system of claim 14, wherein the operations further comprise:
generating a second prompt based on the input dataset, the first prompt structure, the use case, and the evaluation metric;
providing the second prompt to the language model;
receiving a second use case generated by the language model for the input dataset; and
generating a second data pipeline based on the second use case.

### Claim 18 (depends on 17)

The system of claim 17, wherein the use case is a first use case, the output dataset is a first output dataset, and the data pipeline is a first data pipeline;
wherein the operations further comprise:
receiving a second use case generated by the language model for the input dataset, the second use case including a second description of how to use the input dataset;
generating a second data pipeline based on the second use case; and
applying the second data pipeline to the input dataset to generate a second output dataset.

### Claim 19

A method for data pipeline evaluations, the method comprising:
receiving an input dataset, the input dataset including a data schema;
generating a first prompt based on the input dataset and a first prompt structure having one or more text strings and one or more blanks;
providing the first prompt to a language model;
receiving a use case generated by the language model for the input dataset, the use case including a description of how to use the input dataset;
generating a data pipeline based on the use case;
applying the data pipeline to the input dataset to generate an output dataset;
generating a second prompt based on a second prompt structure, the second prompt structure associated with the use case and the output dataset;
providing the second prompt to the language model;
receiving an evaluation function generated by the language model; and
determining an evaluation metric using the evaluation function based on the use case and the output dataset;
wherein the generating a first prompt based on the input dataset and a first prompt structure comprises:
determining text data based on the input dataset; and
filling in the one or more blanks using the determined text data;
wherein the data schema includes one or more data field names and one or more data types;
wherein the determining text data based on the input dataset comprises extracting at least one of the one or more data field names and a corresponding data type from the input dataset;
wherein the method is performed using one or more processors.

### Claim 20 (depends on 19)

The method of claim 19, wherein the language model is a first language model, wherein the generating a data pipeline based on the use case comprises generating the data pipeline using a data pipeline builder including a second language model different from the first language model.
