# US 2021/0279208 A1 · Validating data for integration

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0279208 A1
- Document ID: US20210279208A1
- Status: Published application
- Kind: A1
- Application number: 17/330,250
- Filed: 2021-05-25
- Published: 2021-09-09
- Earliest claimed: 2017-09-05
- Assignees: Palantir Technologies Inc.
- Applicants: Palantir Technologies Inc.
- Inventors: Andrew Higgins; Anshuman Prasad; Harsh Pandey; Luis Ruigomez; Nabeel Qureshi
- Web page: https://patentgrep.com/patent/US20210279208A1

## CPC

- G06F 16/168: {Details of user interfaces specifically adapted to file systems, e.g. browsing and visualisation, 2d or 3d GUIs (query results presentation G06F16/156)}
- G06F 16/2365: {Ensuring data consistency and integrity}
- G06F 16/254: {Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses}
- G06F 40/226: Validation
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Systems, methods, and non-transitory computer readable media are provided for validating data for integration. A file to be integrated into a dataset may be determined. The file may be characterized by a file name and metadata. Whether the dataset can be identified based on the file name and the metadata may be determined. Based on the dataset not being identifiable, information needed to identify the dataset may be requested. Based on the dataset being identifiable, the file may be routed for integration into the dataset.

## Claims

### Claim 1

A method for validating input data for integration into datasets, the method comprising:
identifying a first file designated for integration into at least one dataset;
determining a first file attribute for the first file;
selecting, from among a plurality of validation rules respectively based on different file attribute types, a first validation rule for validating the first file, wherein the first validation rule is selected based on an association between the first validation rule and a first file attribute type of the first file attribute;
validating the first file attribute using the first validation rule, wherein validating the first file attribute comprises determining that one or more validation conditions of the first validation rule are met by the first file attribute;
determining a first dataset into which to integrate the first file based on validation of the first file attribute with respect to the first validation rule; and
routing the first file for integration into the first dataset.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
identifying a second file designated for integration into at least one dataset;
determining a second file attribute for the second file;
selecting, from among the plurality of validation rules respectively based on the different file attribute types, a second validation rule for validating the second file, the second validation rule being based on a second file attribute type of the second attribute, the second file attribute type being different from the first file attribute type;
evaluating the second file attribute against the second validation rule;
determining that the second file attribute satisfies the second validation rule;
determining a second dataset into which to integrate the second file based on the second validation rule being satisfied; and
routing the second file for integration into the second dataset.

### Claim 3 (depends on 2)

The method of claim 2, wherein the first file attribute type is a file name or a file naming convention of the first file and the second file attribute type is metadata of the second file.

### Claim 4 (depends on 3)

The method of claim 3, wherein the metadata is contained in a file name of the second file.

### Claim 5 (depends on 1)

The method of claim 1, wherein the first file attribute type is a file type or a file format of the first file.

### Claim 6 (depends on 1)

The method of claim 1, wherein the first validation rule is a dynamic rule that changes based on information contained in the first file attribute.

### Claim 7 (depends on 1)

The method of claim 1, further comprising modifying the first validation rule based on a user input.

### Claim 8 (depends on 7)

The method of claim 7, wherein the user input is received via a data integration interface.

### Claim 9 (depends on 8)

The method of claim 8, further comprising receiving a selection of the first validation rule via the data integration interface.

### Claim 10 (depends on 1)

The method of claim 1, further comprising:
selecting, from among the plurality of validation rules respectively based on different file attribute types, a second validation rule for validating the first file, the second validation rule being based on a second file attribute type of a second attribute of the first file, the second file attribute type being different from the first file attribute type; and
evaluating the second file attribute against the second validation rule.

### Claim 11 (depends on 10)

The method of claim 10, wherein the second file attribute is evaluated against the second validation rule in a combined processing stage that includes evaluating the first file attribute against the first validation rule.

### Claim 12 (depends on 10)

The method of claim 10, wherein the second file attribute is evaluated against the second validation rule in a different processing stage than the first file attribute is evaluated against the first validation rule.

### Claim 13 (depends on 1)

The method of claim 1, wherein determining the first dataset comprises determining the first dataset based on the first file attribute type and information contained in the first file attribute.

### Claim 14

A system for validating input data for integration into datasets, the system comprising:
at least one memory storing computer-executable instructions; and
at least one processor configured to access the at least one memory and execute the computer-executable instructions to:
identify a first file designated for integration into at least one dataset;
determine a first file attribute for the first file;
select, from among a plurality of validation rules respectively based on different file attribute types, a first validation rule for validating the first file, wherein the first validation rule is selected based on an association between the first validation rule and a first file attribute type of the first file attribute;
validate the first file attribute using the first validation rule, wherein validating the first file attribute comprises determining that one or more validation conditions of the first validation rule are met by the first file attribute;
determine a first dataset into which to integrate the first file based on validation of the first file attribute with respect to the first validation rule; and
route the first file for integration into the first dataset.

### Claim 15 (depends on 14)

The system of claim 14, wherein the at least one processor is further configured to execute the computer-executable instructions to:
identify a second file designated for integration into at least one dataset;
determine a second file attribute for the second file;
select, from among the plurality of validation rules respectively based on the different file attribute types, a second validation rule for validating the second file, the second validation rule being based on a second file attribute type of the second attribute, the second file attribute type being different from the first file attribute type;
evaluate the second file attribute against the second validation rule;
determine that the second file attribute satisfies the second validation rule;
determine a second dataset into which to integrate the second file based on the second validation rule being satisfied; and
route the second file for integration into the second dataset,
wherein the first file attribute type is a file name or a file naming convention of the first file and the second file attribute type is metadata of the second file.

### Claim 16 (depends on 15)

The system of claim 15, wherein the metadata is contained in a file name of the second file.

### Claim 17 (depends on 14)

The system of claim 14, wherein the first file attribute type is a file type or a file format of the first file.

### Claim 18 (depends on 14)

The system of claim 14, wherein the first validation rule is a dynamic rule that changes based on information contained in the first file attribute.

### Claim 19

A computer program product for validating input data for integration into datasets comprising a non-transitory computer-readable medium storing computer-executable instructions that, when executed by a processor, cause operations to be performed comprising:
identifying a first file designated for integration into at least one dataset;
determining a first file attribute for the first file;
selecting, from among a plurality of validation rules respectively based on different file attribute types, a first validation rule for validating the first file, wherein the first validation rule is selected based on an association between the first validation rule and a first file attribute type of the first file attribute;
validating the first file attribute using the first validation rule, wherein validating the first file attribute comprises determining that one or more validation conditions of the first validation rule are met by the first file attribute;
determining a first dataset into which to integrate the first file based on validation of the first file attribute with respect to the first validation rule; and
routing the first file for integration into the first dataset.

### Claim 20 (depends on 19)

The computer program product of claim 19, wherein the first validation rule is a dynamic rule that changes based on information contained in the first file attribute.
