US 2023/0419102 A1Published application

Token synthesis for machine learning models

AssigneeCapital One Services, LLC

InventorsAnh TRUONG, Jeremy GOODSITT, Austin WALTERS

Life of this patent

Estimating the term…

Loading patent…

Abstract

A method includes generating a first vector associated with a token using a first set of parameters of a first learning model based on the token, determining a prediction indicating that the token is associated with a first label based on a set of clustering criteria, the first vector, vectors of a first vector set, and vectors of a second vector. The method includes generating a perturbed vector associated with a second label by modifying a value of the first vector and updating the second vector set to comprise the perturbed vector. The method also includes generating a synthesized token associated with the second vector set based on the perturbed vector using a second set of parameters of the first learning model and training a second learning model based on the synthesized token.

Claims20

Dependent claims

A method for training a classification system to recognize sensitive data by perturbing vectors in a latent space representing sensitive data comprising:

obtaining, from a neural network, a network-based prediction indicating that a candidate token associated with a sensitive data label should have a non-sensitive data label;
encoding, using an encoder layer of an autoencoder, the candidate token to generate a first vector associated with the candidate token;
determining a clustering-based prediction by applying a set of clustering criteria to first distances between the first vector and vectors of a non-sensitive label set in a latent space and second distances between the first vector and vectors of a sensitive label set in the latent space, the clustering-based prediction indicating that the candidate token should have the non-sensitive data label;
in response to the clustering-based prediction not matching the sensitive data label associated with the candidate token, generating perturbed vectors associated with the sensitive data label that are similar to the first vector by randomly selecting multiple values of the first vector for each of the perturbed vectors and applying a random threshold-restricted offset to each of the selected multiple values;
updating the sensitive label set to comprise the perturbed vectors, wherein applying the set of clustering criteria to the first distances and distances between the first vector and the perturbed vectors indicates that the candidate token is associated with the sensitive data label;
decoding, with a decoder layer of the autoencoder, the perturbed vectors to generate synthesized sensitive tokens associated with the sensitive label set;
and
retraining the neural network based on the synthesized sensitive tokens, wherein the neural network associates the candidate token with the sensitive data label after the retraining.

5 more elements

One or more tangible, non-transitory, machine-readable media storing instructions that, when executed by one or more processors, effectuate operations comprising:

generating a first vector in a latent space associated with a token using a first set of neural network layers based on the token;
determining a prediction indicating that the token is associated with a first label based on a set of clustering criteria, the first vector, a first vector set associated with the first label, and a second vector set associated with a second label, wherein the token is associated with a second label in a dataset, and wherein the first and second labels are mutually exclusive with respect to each other;
in response to a determination that the prediction does not match the association between the token and the second label stored in the dataset, generating a perturbed vector associated with the second label by modifying a value of the first vector based on an offset parameter;
updating the second vector set to comprise the perturbed vector, wherein applying the set of clustering criteria to the first vector and the perturbed vector indicates that the token is associated with the second label;
generating a synthesized token associated with the second vector set based on the perturbed vector using a second set of neural network layers;
and
training a machine learning model based on the synthesized token.

4 more elements

A system comprising:

one or more processors;
and
memory storing computer program instructions that, when executed by the one or more processors, cause the one or more processors to effectuate operations comprising:
generating a first vector associated with a token using a first set of parameters of a first machine learning model based on the token;
determining a prediction indicating that the token is associated with a first label based on a set of clustering criteria, the first vector, vectors of a first vector set in a latent space associated with the first label, and vectors of a second vector set in the latent space associated with a second label, wherein the token is associated with the second label in a dataset;
in response to a determination that the prediction indicates that the token is associated with the first label, generating a perturbed vector associated with the second label by modifying a value of the first vector;
updating the second vector set to comprise the perturbed vector, wherein applying the set of clustering criteria to the first vector and the perturbed vector indicates that the token is associated with the second label;
generating a synthesized token associated with the second vector set based on the perturbed vector using a second set of parameters of the first machine learning model;
and
training a second machine learning model based on the synthesized token.

7 more elements

Dependencies are inferred from the claim text.