# US 10,503,574 B1 · Systems and methods for validating data

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,503,574 B1
- Document ID: US10503574B1
- Status: Granted
- Kind: B1
- Application number: 15/670,304
- Filed: 2017-08-07
- Granted: 2019-12-10
- Earliest claimed: 2017-04-10
- Assignees: Palantir Technologies Inc.
- Applicants: Palantir Technologies Inc.
- Inventors: David Lisuk; Guodong Xu; Luis Voloch; Matthew Elkherj
- Web page: https://patentgrep.com/patent/US10503574B1

## CPC

- G06F 11/0709: {in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems}
- G06F 3/048: Interaction techniques based on graphical user interfaces [GUI]
- G06F 16/215: Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
- G06F 16/23: Updating
- G06F 16/43: Querying
- G06F 17/18: for evaluating statistical data {, e.g. average values, frequency distributions, probability functions, regression analysis (forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04)}
- G06Q 10/10: Office automation; Time management
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

Systems and methods are validating data in a data set. A data set including data to validate and a validator to use in validating the data is selected based on user input generated based on interactions of a user with a graphical user interface. The validator is applied to the data to determine whether one or more statistics generated through application of the validator to the data is valid or invalid based on a validation routine associated with the validator. A data quality report indicating whether the data set is valid or invalid, based on a determination of whether the one or more statistics is valid or invalid, is generated and selectively presented to the user through the graphical user interface.

## Claims

### Claim 1

A system for validating data comprising:
one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the system to:
select a data set including data to validate based on user input received from a user in interacting with a graphical user interface;
select a validator including at least one validation parameter to use in validating the data in the data set based on the user input;
apply the validator to the data in the data set to determine whether one or more statistics generated through application of the validator to the data in the data set using the at least one validation parameter is valid according to at least one validation routine associated with the validator;
flag the data set as invalid if at least one of the one or more statistics generated through application of the validator are determined to be invalid according to the at least one validation routine;
generate a data quality report for the data set indicating whether the data set is valid or invalid based on a determination of whether the one or more statistics are valid according to the at least one validation routine;
selectively present the data quality report to the user through the graphical user interface;
re-apply the validator to the data in the data set to determine whether the one or more statistics generated through application of the validator to the data in the data set using the at least one validation parameter are valid according to the at least one validation routine;
generate another data quality report for the data indicating whether the data set is valid or invalid based on a determination made through re-application of the validator to the data in the data set of whether the one or more statistics are valid according to the at least one validation routine; and
selectively present the another data quality report to the user through the graphical user interface.

### Claim 2 (depends on 1)

The system of claim 1, wherein the system is further caused to select the at least one validation parameter based on the user input.

### Claim 3 (depends on 1)

The system of claim 1, wherein the system is further caused to:
select the at least one validation routine based on the user input;
customize the validator by associating the validator with the at least one validation routine selected based on the user input for use in applying the validator to the data in the data set to determine whether the one or more statistics are valid according to the at least one validation routine.

### Claim 4 (depends on 1)

The system of claim 1, wherein the system is further caused to:
select the at least one validation parameter based on the user input;
customize the validator by associating the validation parameters with the validator according to the user input.

### Claim 5 (depends on 1)

The system of claim 1, wherein the system is further caused to:
receive additional user input;
modify the validator to create a modified validator by modifying the at least one validation parameter to create at least one modified validation parameter in the modified validator according to the additional user input;
apply the modified validator to the data in the data set to determine whether an additional one or more statistics generated through application of the modified validator to the data in the data set using the at least one modified validation parameter are valid according to the at least one validation routine;
generate another data quality report for the data set indicating whether the data set is valid or invalid based on a determination of whether the additional one or more statistics are valid according to the at least one validation routine; and
selectively present the another data quality report to the user through the graphical user interface.

### Claim 6 (depends on 1)

The system of claim 1, wherein the system is further caused to:
receive additional user input;
customize the validator to create a customized validator by modifying the at least one validation routine to create at least one modified validation routine in the customized validator according to the additional user input;
apply the customized validator to the data in the data set to determine whether an additional one or more statistics generated through application of the customized validator to the data in the data set using the at least one validation parameter are valid according to the at least one modified validation routine;
generate another data quality report for the data set indicating whether the data set is valid or invalid based on a determination of whether the additional one or more statistics are valid according to the at least one modified validation routine; and
selectively present the another data quality report to the user through the graphical user interface.

### Claim 7 (depends on 1)

The system of claim 1, wherein the system is further caused to re-apply the validator to the data in the data set based on one or a combination of re-application instructions received from a user, a passage of a specific amount of time, and a change made to the data in the data set.

### Claim 8 (depends on 1)

The system of claim 1, wherein the system is further caused to:
select another data set including additional data to validate using the validator based on the user input;
concurrently with application of the validator to the data in the data set, apply the validator to the additional data in the another data set to determine whether one or more additional statistics generated through application of the validator to the additional data in the another data set using the at least one validation parameter is valid according to the at least one validation routine associated with the validator;
generate another data quality report for the data set and the another data set indicating whether either or both the data set and the another data set are valid or invalid based on a determination of whether either or both the one or more statistics and the one or more additional statistics are valid according to the at least one validation routine; and
selectively present the another data quality report to the user through the graphical user interface.

### Claim 9 (depends on 1)

The system of claim 1, wherein the system is further caused to:
select another validator including at least one another validation parameter and at least one another validation routine to use in validating the data in the data set based on the user input;
concurrently with application of the validator to the data in the data set, apply the another validator to the data in the data set to determine whether one or more additional statistics generated through application of the another validator to the data in the data set using the at least one another validator parameter are valid according to the at least one another validation routine associated with the another validator;
generate another data quality report for the data set indicating whether the data set is valid or invalid based on either or both the determination of whether the one or more statistics are valid according to the at least one validation routine and a determination of whether the one or more additional statistics are valid according to the at least one another validation routine; and
selectively present the another data quality report to the user through the graphical user interface.

### Claim 10

A method being implemented by a computing system including one or more physical processors and storage media storing machine-readable instructions, the method comprising:
selecting a data set including data to validate based on user input received from a user in interacting with a graphical user interface;
selecting a validator including at least one validation parameter to use in validating the data in the data set based on the user input;
applying the validator to the data in the data set to determine whether one or more statistics generated through application of the validator to the data in the data set using the at least one validation parameter is valid according to at least one validation routine associated with the validator;
flagging the data set as invalid if at least one of the one or more statistics generated through application of the validator are determined to be invalid according to the at least one validation routine;
generating a data quality report for the data set indicating whether the data set is valid or invalid based on a determination of whether the one or more statistics are valid according to the at least one validation routine;
selectively presenting the data quality report to the user through the graphical user interface;
re-applying the validator to the data in the data set to determine whether the one or more statistics generated through application of the validator to the data in the data set using the at least one validation parameter are valid according to the at least one validation routine;
generating another data quality report for the data indicating whether the data set is valid or invalid based on a determination made through re-application of the validator to the data in the data set of whether the one or more statistics are valid according to the at least one validation routine; and
selectively presenting the another data quality report to the user through the graphical user interface.

### Claim 11 (depends on 10)

The method of claim 10, further comprising selecting the at least one validation parameter based on the user input.

### Claim 12 (depends on 10)

The method of claim 10, further comprising:
selecting the at least one validation routine based on the user input;
customizing the validator by associating the validator with the at least one validation routine selected based on the user input for use in applying the validator to the data in the data set to determine whether the one or more statistics are valid according to the at least one validation routine.

### Claim 13 (depends on 10)

The method of claim 10, further comprising:
selecting the at least one validation parameter based on the user input;
customizing the validator by associating the validation parameters with the validator according to the user input.

### Claim 14 (depends on 10)

The method of claim 10, further comprising:
receiving additional user input;
modifying the validator to create a modified validator by modifying the at least one validation parameter to create at least one modified validation parameter in the modified validator according to the additional user input;
applying the modified validator to the data in the data set to determine whether an additional one or more statistics generated through application of the modified validator to the data in the data set using the at least one modified validation parameter are valid according to the at least one validation routine;
generating another data quality report for the data set indicating whether the data set is valid or invalid based on a determination of whether the additional one or more statistics are valid according to the at least one validation routine; and
selectively presenting the another data quality report to the user through the graphical user interface.

### Claim 15 (depends on 10)

The method of claim 10, further comprising:
receiving additional user input;
customizing the validator to create a customized validator by modifying the at least one validation routine to create at least one modified validation routine in the customized validator according to the additional user input;
applying the customized validator to the data in the data set to determine whether an additional one or more statistics generated through application of the customized validator to the data in the data set using the at least one validation parameter are valid according to the at least one modified validation routine;
generating another data quality report for the data set indicating whether the data set is valid or invalid based on a determination of whether the additional one or more statistics are valid according to the at least one modified validation routine; and
selectively presenting the another data quality report to the user through the graphical user interface.

### Claim 16 (depends on 10)

The method of claim 10, further comprising:
selecting another data set including additional data to validate using the validator based on the user input;
concurrently with application of the validator to the data in the data set, applying the validator to the additional data in the another data set to determine whether one or more additional statistics generated through application of the validator to the additional data in the another data set using the at least one validation parameter is valid according to the at least one validation routine associated with the validator;
generating another data quality report for the data set and the another data set indicating whether either or both the data set and the another data set are valid or invalid based on a determination of whether either or both the one or more statistics and the one or more additional statistics are valid according to the at least one validation routine; and
selectively presenting the another data quality report to the user through the graphical user interface.

### Claim 17 (depends on 10)

The method of claim 10, further comprising:
selecting another validator including at least one another validation parameter and at least one another validation routine to use in validating the data in the data set based on the user input;
concurrently with application of the validator to the data in the data set, applying the another validator to the data in the data set to determine whether one or more additional statistics generated through application of the another validator to the data in the data set using the at least one another validator parameter are valid according to the at least one another validation routine associated with the another validator;
generating another data quality report for the data set indicating whether the data set is valid or invalid based on either or both the determination of whether the one or more statistics are valid according to the at least one validation routine and a determination of whether the one or more additional statistics are valid according to the at least one another validation routine; and
selectively presenting the another data quality report to the user through the graphical user interface.

### Claim 18

A non-transitory computer readable medium comprising instructions that, when executed, cause one or more processors to perform:
selecting a data set including data to validate based on user input received from a user in interacting with a graphical user interface;
selecting a validator including at least one validation parameter to use in validating the data in the data set based on the user input;
applying the validator to the data in the data set to determine whether one or more statistics generated through application of the validator to the data in the data set using the at least one validation parameter is valid according to at least one validation routine associated with the validator;
flagging the data set as invalid if at least one of the one or more statistics generated through application of the validator are determined to be invalid according to the at least one validation routine;
generating a data quality report for the data set indicating whether the data set is valid or invalid based on a determination of whether the one or more statistics are valid according to the at least one validation routine;
selectively presenting the data quality report to the user through the graphical user interface;
re-applying the validator to the data in the data set to determine whether the one or more statistics generated through application of the validator to the data in the data set using the at least one validation parameter are valid according to the at least one validation routine;
generating another data quality report for the data indicating whether the data set is valid or invalid based on a determination made through re-application of the validator to the data in the data set of whether the one or more statistics are valid according to the at least one validation routine; and
selectively presenting the another data quality report to the user through the graphical user interface.