US 2026/0093674 A1 · QUERY-BASED EVALUATION OF ERROR PATTERNS IN ANNOTATED TRAINING DATA FOR MACHINE LEARNING SYSTEMS AND APPLICATIONS

Abstract

In various examples, query-based evaluation of error patterns in annotated training data for machine learning systems and applications is provided. An error pattern evaluator may include an error pattern extraction function and an error pattern search algorithm. An annotated validation sample associated with an inaccurate prediction from a machine learning model may be analyzed to extract an error pattern. A set of annotated training data used for training the machine learning model may be analyzed by the error pattern extraction function to generate an error pattern for individual samples of annotated training data to define a search dataset. An error pattern search algorithm may use the error pattern derived from the inaccurate prediction as a query to search the dataset to identify each occurrence of samples having an error pattern similar to the query error pattern. The set of annotated training data may be corrected based on the search results.

Claims21

Dependent claims

One or more processors comprising processing circuitry to:

detect one or more prediction errors from an output of a machine learning model, the one or more prediction errors being generated based at least on an input of a first data sample to the machine learning model;
extract a first annotation error pattern from the first data sample;
extract one or more second annotation error patterns from a dataset of second data samples that were used to train the machine learning model;
generate, based at least on applying a query input determined using the first annotation error pattern to a search algorithm, a search result based at least on a search of the dataset of second data samples using the query input;
modify the dataset of second data samples based at least on the search result to produce a remediated dataset of training data;
and
update one or more parameters of the machine learning model based at least on a training process that uses the remediated dataset of training data.

4 more elements

Dependencies are inferred from the claim text.

Query-based evaluation of error patterns in annotated training data for machine learning systems and applications

Life of this patent

Query-based evaluation of error patterns in annotated training data for machine learning systems and applications

Life of this patent

Abstract

Claims21