US 2025/0028910 A1 · Method for Filtering and Semi-Automatically Labeling Training Data

Abstract

A method is provided for efficiently providing sentiments or other manual labels for textual training data. The method includes using an embedding model to project acquired user text to an embedding vector in an embedding space. Distances (e.g., cosine similarities) between this embedding vector and the embedding vectors determined for a plurality of already-label user text training examples are then determined. The already-labeled user text that has the shortest distance is determined and the label thereof is prospectively applied to the acquired user text and presented to a user for approval. The user can approve the prospectively applied label, in which case the newly acquired text is added to the training data with the prospectively applied label associated therewith for later use in training a language model. Alternatively, the user can decline the prospectively applied label and apply an alternative label to the newly acquired text.

Claims20

Dependent claims

A method comprising:

obtaining user text;
applying an embedding model to the user text to generate an embedding vector in a vector space, wherein the embedding model was trained to generate respective embedding vectors for training data sets, and wherein the training data sets include a plurality of textual training examples each with respectively associated class labels;
identifying a particular textual training example, of the plurality of textual training examples, whose associated embedding vector is closest in the vector space to the embedding vector that is associated with the user text;
providing an indication of the user text and a class label respectively associated with the particular textual training example;
receiving a response that the class label correctly classifies the user text;
and
in response to receiving the response, training a production model using the user text associated with the class label.

4 more elements

A non-transitory computer-readable medium, having stored thereon program instructions that, upon execution by a computing system, cause the computing system to perform operations comprising:

obtaining user text;
applying an embedding model to the user text to generate an embedding vector in a vector space, wherein the embedding model was trained to generate respective embedding vectors for training data sets, and wherein the training data sets include a plurality of textual training examples each with respectively associated class labels;
identifying a particular textual training example, of the plurality of textual training examples, whose associated embedding vector is closest in the vector space to the embedding vector that is associated with the user text;
providing an indication of the user text and a class label respectively associated with the particular textual training example;
receiving a response that the class label correctly classifies the user text;
and
in response to receiving the response, training a production model using the user text associated with the class label.

4 more elements

A system comprising:

one or more processors;
and
memory, containing program instructions that, upon execution by the one or more processors, cause the system to perform operations comprising:
obtaining user text;
applying an embedding model to the user text to generate an embedding vector in a vector space, wherein the embedding model was trained to generate respective embedding vectors for training data sets, and wherein the training data sets include a plurality of textual training examples each with respectively associated class labels;
identifying a particular textual training example, of the plurality of textual training examples, whose associated embedding vector is closest in the vector space to the embedding vector that is associated with the user text;
providing an indication of the user text and a class label respectively associated with the particular textual training example;
receiving a response that the class label correctly classifies the user text;
and
in response to receiving the response, training a production model using the user text associated with the class label.

7 more elements

Dependencies are inferred from the claim text.

Method for Filtering and Semi-Automatically Labeling Training Data

Life of this patent

Method for Filtering and Semi-Automatically Labeling Training Data

Life of this patent

Abstract

Claims20