US 2024/0202571 A1Published application

Systems and methods for label selection for machine learning model training based on dataset similiarity metrics

AssigneeCapital One Services, LLC

InventorsHaytham YAGHI, Tania Cruz Morales, Shannon Yogerst

Life of this patent

Estimating the term…

Loading patent…

Abstract

Systems and methods for selecting labels based on dataset similarity for machine learning model training data are disclosed herein. The system may receive a label modification request for a training datum. The system may retrieve label records corresponding to the datum identifier. The system may compare a pre-existing label with the new label. Based on determining that the pre-existing label differs from the new label, the system may generate a first dataset and a second dataset. The system may calculate a first average similarity metric and a second average similarity metric. The system may compare the first average similarity metric with the second average similarity metric. The system may generate a recommendation for a modified label.

Claims20

Dependent claims

A system for selecting labels based on dataset similarity for machine learning model training chatbot text data comprising:

one or more processors;
and
a non-transitory, computer-readable medium comprising instructions that when executed by the one or more processors cause operations comprising:
receiving, at a device on a computer network, a label modification request for a training datum for a machine learning model, wherein the label modification request comprises a new label and a datum identifier, and wherein the new label comprises a category of chatbot data;
retrieving, from a label record database, a plurality of label records corresponding to the datum identifier, wherein the plurality of label records comprises a plurality of pre-existing labels previously applied to the training datum;
comparing a pre-existing label from the plurality of pre-existing labels with the new label;
based on determining that the pre-existing label from the plurality of pre-existing labels differs from the new label, generating, based on the label record database, (1) a first dataset labeled with the new label and (2) a second dataset labeled with the pre-existing label;
calculating a first plurality of similarity metrics between the training datum and each datum of the first dataset and a second plurality of similarity metrics between the training datum and each datum of the second dataset;
calculating a first average similarity metric based on an average of the first plurality of similarity metrics and a second average similarity metric based on an average of the second plurality of similarity metrics;
comparing the first average similarity metric with the second average similarity metric;
and
based on determining that the second average similarity metric is higher than the first average similarity metric, generating a recommendation for a modified label based on the pre-existing label.

9 more elements

A method for selecting labels based on dataset similarity for machine learning model training data, the method comprising:

receiving, at a device on a computer network, a label modification request for a training datum for a machine learning model, wherein the label modification request comprises a new label and a datum identifier;
retrieving, from a label record database, a plurality of label records corresponding to the datum identifier,
comparing a pre-existing label from the plurality of label records with the new label;
based on determining that the pre-existing label from the plurality of label records differs from the new label, generating, based on the label record database, (1) a first dataset labeled with the new label and (2) a second dataset labeled with the pre-existing label;
calculating (1) a first average similarity metric between the training datum and the first dataset and (2) a second average similarity metric between the training datum and the second dataset;
comparing the first average similarity metric with the second average similarity metric;
and
based on comparing the first average similarity metric with the second average similarity metric, generating a recommendation for a modified label.

5 more elements

A non-transitory, computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

receiving, at a device on a computer network, a label modification request for a training datum for a machine learning model, wherein the label modification request comprises a new label and a datum identifier,
retrieving, from a label record database, a plurality of label records corresponding to the datum identifier,
comparing a pre-existing label from the plurality of label records with the new label;
based on determining that the pre-existing label from the plurality of label records differs from the new label, generating, based on the label record database, (1) a first dataset labeled with the new label and (2) a second dataset labeled with the pre-existing label;
calculating (1) a first average similarity metric between the training datum and the first dataset and (2) a second average similarity metric between the training datum and the second dataset;
comparing the first average similarity metric with the second average similarity metric;
and
based on comparing the first average similarity metric with the second average similarity metric, generating a recommendation for a modified label.

5 more elements

Dependencies are inferred from the claim text.