# US 2024/0004849 A1 · Systems and methods for continuous data profiling

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0004849 A1
- Document ID: US20240004849A1
- Status: Published application
- Kind: A1
- Application number: 18/469,363
- Filed: 2023-09-18
- Published: 2024-01-04
- Earliest claimed: 2021-06-30
- Assignees: Collibra Belgium BV
- Applicants: Collibra Belgium BV
- Inventors: James B. Cushman, II; Vadim Vaks; Satyender Goel
- Web page: https://patentgrep.com/patent/US20240004849A1

## CPC

- G06F 16/215: Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
- G06F 16/2282: {Tablespace storage structures; Management thereof}
- G06F 16/24565: {Triggers; Constraints}
- G06F 16/2457: with adaptation to user needs
- G06F 16/252: {between a Database Management System and a front-end application}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

The present disclosure is directed to continuous data profiling (CDP). Entities may house large amounts of disorganized and/or duplicative data. To organize and standardize data across a data set, the data may be profiled. However, profiling large data set can be inefficient and give rise to security problems, as profiling datasets typically requires exporting a dataset to a third-party profiling runtime environment. To remedy these issues, the present disclosure is directed to a continuous data profiling platform that comprises a CDP manager communicatively coupled to a client's database. The CDP manager provides access to a CDP API that may install CDP tools on a client's native database environment, enabling the database management system to profile datasets within the client's native database environment, which results in a more efficient use of computing resources and more secure process of profiling datasets.

## Claims

### Claim 1

A method of continuously profiling data, the method comprising:
receiving at least one input stream of data;
profiling the at least one input stream of data by,
identifying at least one attribute in the at least one input stream of data, wherein the at least one attribute is associated with a series of features, and
determining a profiling score for the at least one attribute based on an aggregation of the series of features;
generating a profiled set of data based on the profiling of the at least one input stream of data, wherein the profiled set of data includes the profiling score for the at least one attribute; and
storing the profiled set of data in at least one client database.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
retrieving at least one set of profiling rules and at least one set of processing rules that correspond to the at least one attribute;
comparing the at least one attribute with the at least one set of profiling rules and the at least one set of processing rules to validate information included in the at least one attribute;
in response to determining that the information included in the at least one attribute is profiled according to the at least one set of processing rules, storing the information for the at least one attribute into at least one profiled format according to the at least one set of processing rules, wherein the information is provisioned via at least one API gateway; and
processing the at least one attribute through at least one set of rules engines.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
in response to determining that the at least one attribute is indicative of a name, processing the at least one attribute through a name engine that associates the at least one attribute with associated names included in a listing of associated names.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
in response to determining that the at least one attribute is indicative of an address, processing the at least one attribute through an address library engine that adds the at least one attribute to a library of addresses.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
receiving at least one set of continuous data profiling tools from a continuous data profiling manager, wherein the continuous data profiling manager is a frontend software application communicatively coupled to the at least one client database.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
connecting at least one continuous data profiling (CDP) manager application to the at least one client database; and
receiving at least one instruction or at least one function via the at least one continuous data profiling (CDP) manager application.

### Claim 7 (depends on 6)

The method of claim 6,
wherein the at least one function is a first function for generating a statistical table of profiled data,
wherein the at least one function is a second function for generating a change-data-store table of profiled data,
wherein the at least one function is a third function for managing at least one profiling process, and
wherein the at least one function is a fourth function for managing at least one profile trigger that is compared to the at least one input stream of data.

### Claim 8

A system comprising:
one or more processors; and
one or more memories storing instructions that, when executed by the one or more processors, cause the system to perform a process for continuously profiling data, the process comprising:
receiving at least one input stream of data;
profiling the at least one input stream of data by,
identifying at least one attribute in the at least one input stream of data, wherein the at least one attribute is associated with a series of features, and
determining a profiling score for the at least one attribute based on an aggregation of the series of features;
generating a profiled set of data based on the profiling of the at least one input stream of data, wherein the profiled set of data includes the profiling score for the at least one attribute; and
storing the profiled set of data in at least one client database.

### Claim 9 (depends on 8)

The system according to claim 8, wherein the process further comprises:
retrieving at least one set of profiling rules and at least one set of processing rules that correspond to the at least one attribute;
comparing the at least one attribute with the at least one set of profiling rules and the at least one set of processing rules to validate information included in the at least one attribute;
in response to determining that the information included in the at least one attribute is profiled according to the at least one set of processing rules, storing the information for the at least one attribute into at least one profiled format according to the at least one set of processing rules, wherein the information is provisioned via at least one API gateway; and
processing the at least one attribute through at least one set of rules engines.

### Claim 10 (depends on 8)

The system according to claim 8, wherein the process further comprises:
in response to determining that the at least one attribute is indicative of a name, processing the at least one attribute through a name engine that associates the at least one attribute with associated names included in a listing of associated names.

### Claim 11 (depends on 8)

The system according to claim 8, wherein the process further comprises:
in response to determining that the at least one attribute is indicative of an address, processing the at least one attribute through an address library engine that adds the at least one attribute to a library of addresses.

### Claim 12 (depends on 8)

The system according to claim 8, wherein the process further comprises:
receiving at least one set of continuous data profiling tools from a continuous data profiling manager, wherein the continuous data profiling manager is a frontend software application communicatively coupled to the at least one client database.

### Claim 13 (depends on 8)

The system according to claim 8, wherein the process further comprises:
connecting at least one continuous data profiling (CDP) manager application to the at least one client database; and
receiving at least one instruction or at least one function via the at least one continuous data profiling (CDP) manager application.

### Claim 14 (depends on 13)

The system according to claim 13,
wherein the at least one function is a first function for generating a statistical table of profiled data,
wherein the at least one function is a second function for generating a change-data-store table of profiled data,
wherein the at least one function is a third function for managing at least one profiling process, and
wherein the at least one function is a fourth function for managing at least one profile trigger that is compared to the at least one input stream of data.

### Claim 15

A non-transitory computer-readable medium storing instructions that, when executed by a computing system, cause the computing system to perform operations for continuously profiling data, the operations comprising:
receiving at least one input stream of data;
profiling the at least one input stream of data by,
identifying at least one attribute in the at least one input stream of data, wherein the at least one attribute is associated with a series of features, and
determining a profiling score for the at least one attribute based on an aggregation of the series of features;
generating a profiled set of data based on the profiling of the at least one input stream of data, wherein the profiled set of data includes the profiling score for the at least one attribute; and
storing the profiled set of data in at least one client database.

### Claim 16 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein the operations further comprise:
retrieving at least one set of profiling rules and at least one set of processing rules that correspond to the at least one attribute;
comparing the at least one attribute with the at least one set of profiling rules and the at least one set of processing rules to validate information included in the at least one attribute;
in response to determining that the information included in the at least one attribute is profiled according to the at least one set of processing rules, storing the information for the at least one attribute into at least one profiled format according to the at least one set of processing rules, wherein the information is provisioned via at least one API gateway; and
processing the at least one attribute through at least one set of rules engines.

### Claim 17 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein the operations further comprise:
in response to determining that the at least one attribute is indicative of a name, processing the at least one attribute through a name engine that associates the at least one attribute with associated names included in a listing of associated names.

### Claim 18 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein the operations further comprise:
in response to determining that the at least one attribute is indicative of an address, processing the at least one attribute through an address library engine that adds the at least one attribute to a library of addresses.

### Claim 19 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein the operations further comprise:
receiving at least one set of continuous data profiling tools from a continuous data profiling manager, wherein the continuous data profiling manager is a frontend software application communicatively coupled to the at least one client database.

### Claim 20 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein the operations further comprise:
connecting at least one continuous data profiling (CDP) manager application to the at least one client database; and
receiving at least one instruction or at least one function via the at least one continuous data profiling (CDP) manager application,
wherein the at least one function is a first function for generating a statistical table of profiled data,
wherein the at least one function is a second function for generating a change-data-store table of profiled data,
wherein the at least one function is a third function for managing at least one profiling process, and
wherein the at least one function is a fourth function for managing at least one profile trigger that is compared to the at least one input stream of data.
