US 12,182,089 B2Granted

Systems and methods for storing versioned data while labeling data for artificial intelligence model development

AssigneeCapital One Services, LLC

InventorsTania Cruz Morales, Purva Shanker, Shannon Yogerst, Ignacio Espino, Dan Lin, Nathan Wolfe

Life of this patent

Estimating the term…

Loading patent…

Abstract

Systems and methods are described for maintaining bifurcated data management while labeling data for artificial intelligence model development. For example, the system may receive a first label for a first sample from a first dataset, wherein the first dataset is accessible to a first subset of a plurality of users, and wherein the first subset comprises a first attribute. The system may receive first version metadata of the first label, wherein the first version metadata comprises a proposed label for the first sample assigned by a first user. The system may determine, based on a first user input from the first user, a first grouping of source code files for storing the first version metadata, wherein the first grouping of source code files is accessible to a second subset of the plurality of users, and wherein the second subset comprises a second attribute.

Claims18

Dependent claims

A system for storing versioned data while labeling data for artificial intelligence model development, the system comprising:

a first datastore, wherein the first datastore is accessible to a first subset of a plurality of users, wherein the first datastore comprises a relational database that stores a plurality of samples;
a second datastore, wherein the second datastore is specific to a first workspace, wherein the first workspace is accessible to a second subset of the plurality of users for assigning category labels to samples, and wherein the second datastore comprises a labeled data archive specific to the first workspace;
a third datastore, wherein the third datastore is accessible to a third subset of the plurality of users, and wherein the third datastore comprises unlabeled data sourced from the first datastore;
one or more preprocessors;
and
a non-transitory computer readable medium comprising instructions that when executed by the one or more preprocessors cause operations comprising:
receiving a first label for a first sample from the first datastore;
receiving first version metadata of the first label, wherein the first version metadata comprises a proposed label for the first sample assigned by a first user, and wherein the first version metadata comprises information about a category within which the first label belongs and a confidence in the category;
determining to store the first version metadata in the second datastore, wherein the second datastore is accessible to a second subset of the plurality of users, and wherein the first version metadata is assigned to a two-dimensional database;
receiving a second user input to generate training data for an artificial intelligence model based on version metadata of labels in the second datastore that have a threshold consistency and confidence;
retrieving the first version metadata of labels from the second datastore;
comparing the first version metadata of labels for consistency and confidence, wherein the consistency indicates that the first version metadata of labels, within the category labels, are consistent with each other;
determining that the first version metadata of labels have a threshold level of consistency and confidence;
in response to determining that the first version metadata of labels have the threshold level of consistency, generating a second dataset for training the artificial intelligence model based on the first version metadata of the labels in the second datastore;
and
generating for display, in a user interface, a status notification for generation of the second dataset.

13 more elements

A method for storing versioned data while labeling data for artificial intelligence model development, the method comprising:

receiving a first label for a first sample from a first datastore, wherein the first datastore is accessible to a first subset of a plurality of users to assign category labels to samples, and wherein the first datastore comprises a relational database;
receiving first version metadata of the first label, wherein the first version metadata comprises information about a category within which the first label belongs and a confidence in the category;
determining a second datastore for storing the first version metadata, wherein the second datastore is accessible to a second subset of the plurality of users;
receiving a second user input to generate training data for an artificial intelligence model based on version metadata of labels in the second datastore that have a threshold consistency and confidence;
retrieving the first version metadata of labels from the second datastore;
comparing the first version metadata of labels for consistency, wherein the consistency indicates that the first version metadata of labels, within the category labels, are consistent with each other;
determining that the first version metadata of labels have a threshold level of consistency;
in response to the second user input, generating a second dataset for training the artificial intelligence model based on the first version metadata of the labels in the second datastore that have the threshold consistency;
and
generating for display, in a user interface, a status notification for generation of the second dataset.

7 more elements

A non-transitory computer readable medium comprising instructions that when executed by one or more preprocessors cause operations comprising:

receiving a first label for a first sample from a first datastore, wherein the first datastore is accessible to a first subset of a plurality of users to assign category labels to samples, and wherein the first datastore comprises a relational database;
receiving first version metadata of the first label, wherein the first version metadata comprises information about a category within which the first label belongs;
determining a second datastore for storing the first version metadata, wherein the second datastore is accessible to a second subset of the plurality of users;
receiving a second user input to generate training data for an artificial intelligence model based on version metadata of labels in the second datastore that have a threshold consistency;
retrieving the first version metadata of labels from the second datastore;
comparing the first version metadata of labels for consistency and confidence, wherein the consistency indicates that the first version metadata of labels, within the category labels, are consistent with each other;
determining that the first version metadata of labels have a threshold level of consistency and confidence;
in response to the second user input, generating a second dataset for training the artificial intelligence model based on the first version metadata of the labels in the second datastore that have the threshold consistency and confidence;
and
generating for display, in a user interface, a status notification for generation of the second dataset.

7 more elements

Dependencies are inferred from the claim text.

Facts

Application: 18/156,213Filed 18 Jan 2023
Published: US 2024/0241871 A118 Jul 2024
Granted: 31 Dec 2024
Term ends: Not estimatedThe source records don’t support an estimate.

Classification5 codes

G06F 16/219
Design, administration or maintenance of databases: managing data history or versioning
G06F 16/2474
Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries: sequence data queries, e.g. querying versioned data
G06N 20/00
Machine learning
G06F
Computing and data processing

Family

US 2024/0241871 A1Pre-grant publication · published 18 Jul 2024
US 12,182,089 B2This patent

From the US filing history. Foreign family members aren’t listed.

Timeline

Citations

Open elsewhere

Building an agent or pulling many patents? Use the API or MCP server instead of scraping pages. Get an API key