US 11,636,389 B2 · System and method for improving machine learning models by detecting and removing inaccurate training data

Abstract

Methods, systems and computer program products are described to improve machine learning (ML) model-based classification of data items by identifying and removing inaccurate training data. Inaccurate training samples may be identified, for example, based on excessive variance in vector space between a training sample and a mean of category training samples, and based on a variance between an assigned category and a predicted category for a training sample. Suspect or erroneous samples may be selectively removed based on, for example, vector space variance and/or prediction confidence level. As a result, ML model accuracy may be improved by training on a more accurate revised training set. ML model accuracy may (e.g., also) be improved, for example, by identifying and removing suspect categories with excessive (e.g., weighted) vector space variance. Suspect categories may be retained or revised. Users may (e.g., also) specify a prediction confidence level and/or coverage (e.g., to control accuracy).

Claims20

Dependent claims

A method, comprising:

receiving a training set comprising training samples, including a first training sample, associated or labeled with a first category to train a first machine learning (ML) model to predict whether an inference sample is associated with the first category, where a sample comprises a vectorized data item;
and
improving prediction accuracy of the first ML model by identifying the first training sample as an erroneous sample based on:
a first variance in vector space between the first training sample and a category mean comprising an average of the training samples;
and
a second variance between the first category and a predicted category for the first training sample.

3 more elements

A system, comprising:

at least one processor circuit;
and
at least one memory that stores program code configured to be executed by the at least one processor circuit, the program code comprising:
a data fetcher configured to receive a training set comprising training samples, including a first training sample, associated or labeled with a first category to train a first machine learning (ML) model to predict whether an inference sample is associated with the first category, where a sample comprises a vectorized data item;
and
an artificial intelligence (AI) engine configured to:
identify the first training sample as an erroneous sample based on:
a first variance in vector space between the first training sample and a category mean comprising an average of the training samples;
and
a second variance between the first category and a predicted category for the first training sample.

7 more elements

A computer-readable storage medium having program instructions recorded thereon that, when executed by at least one processor of a computing device, perform a method, the method comprising:

receiving a training set comprising training samples, including a first training sample, associated or labeled with a first category to train a first machine learning (ML) model to predict whether an inference sample is associated with the first category, where a sample comprises a vectorized data item;
and
improving prediction accuracy of the first ML model by identifying the first training sample as an erroneous sample based on:
a first variance in vector space between the first training sample and a category mean comprising an average of the training samples;
and
a second variance between the first category and a predicted category for the first training sample, wherein the second variance comprises the predicted category as a second category, different from the first category, and a confidence level in an accuracy of the predicted category;
selectively removing at least one identified erroneous sample from the training set to create a revised training set;
and
training the first ML model with the revised training set instead of the training set.

6 more elements

Dependencies are inferred from the claim text.

System and method for improving machine learning models by detecting and removing inaccurate training data

Life of this patent

System and method for improving machine learning models by detecting and removing inaccurate training data

Life of this patent

Abstract

Claims20