# US 2024/0005153 A1 · Systems and methods for synthetic data generation using a classifier

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0005153 A1
- Document ID: US20240005153A1
- Status: Published application
- Kind: A1
- Application number: 18/368,318
- Filed: 2023-09-14
- Published: 2024-01-04
- Earliest claimed: 2019-11-18
- Assignees: Capital One Services, LLC
- Applicants: Capital One Services, LLC
- Inventors: Anh TRUONG; Austin WALTERS; Jeremy GOODSITT
- Web page: https://patentgrep.com/patent/US20240005153A1

## CPC

- G06N 3/08: Learning methods
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/0475: Generative networks
- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- G06N 3/0895: Weakly supervised learning, e.g. semi-supervised or self-supervised learning
- G06N 3/09: Supervised learning
- G06N 3/0985: Hyperparameter optimisation; Meta-learning; Learning-to-learn
- G06N 5/04: Inference or reasoning models
- G06N 3/045: Combinations of networks
- G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
- G06N 20/00: Machine learning
- G06N 20/10: using kernel methods, e.g. support vector machines [SVM]
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

The present disclosure provides systems and methods for synthetic data generation. A recurrent neural network can be trained for synthetic data generation by obtaining a sequence of elements and determining, using a classifier, that the sequence corresponds to a token. In response to the determination, a recurrent neural network configured to use a first vocabulary including the elements can be modified to use a second vocabulary, the second vocabulary including the token and the first vocabulary. The modified recurrent neural network can be trained using the token and the sequence of elements. The trained recurrent neural network can be used to generate synthetic data. A classifier can detect sequences of elements in the synthetic data corresponding to tokens. The tokens can replace the sequences of elements in the generated synthetic data and can be provided to the trained recurrent neural network to continue synthetic data generation.

## Claims

### Claim at position 1

1-20. (canceled)

### Claim 21

A computer-implemented method of training a recurrent neural network for synthetic data generation, comprising:
obtaining a sequence of elements;
determining, using a classifier trained to identify matches to a token, that at least a portion of the sequence of elements matches the token; and
in response to the determination:
modifying a recurrent neural network (RNN) to use a second vocabulary, wherein the second vocabulary includes the token and a first vocabulary; and
training the modified RNN using the token, the training comprising reverting the modified RNN to a state associated with the at least a portion of the sequence of elements matching the token.

### Claim 22 (depends on 21)

The computer-implemented method of claim 21, wherein the classifier is configured to identify the sequence of elements as matching a string.

### Claim 23 (depends on 22)

The computer-implemented method of claim 22, wherein the sequence of elements is identical in composition to the elements in the string.

### Claim 24 (depends on 21)

The computer-implemented method of claim 21, wherein modifying the RNN to comprises augmenting the RNN with one or more weights associated with the token.

### Claim 25 (depends on 21)

The computer-implemented method of claim 21, wherein the classifier is trained using training data that is labeled to indicate the existence of one or more strings.

### Claim 26 (depends on 21)

The computer-implemented method of claim 21, wherein the classifier is trained using training data that comprises key-value pairs.

### Claim 27 (depends on 21)

The computer-implemented method of claim 21, wherein at least one of the first vocabulary or the second vocabulary includes strings from database records.

### Claim 28 (depends on 21)

The computer-implemented method of claim 21, wherein:
training the modified RNN comprises a forward phase and backward phase; and
the forward phase for training comprises sequentially inputting each element in a training sequence to the RNN.

### Claim 29 (depends on 28)

The computer-implemented method of claim 28, wherein:
training the modified RNN using the token comprises a forward phase and backward phase; and
the backward phase comprises updating one or more weights of the RNN using a backpropagation-through-time algorithm.

### Claim 30

A system for generating synthetic data, comprising:
at least one processor; and
at least one memory containing instructions that, when executed by the at least one processor, cause the system to perform operations comprising:
obtaining a sequence of elements;
determining, using a classifier trained to identify matches to a token, that at least a portion of the sequence of elements matches the token; and
in response to the determination:
modifying a recurrent neural network (RNN) to use a second vocabulary, wherein the second vocabulary includes at least one of the token or the first vocabulary; and
training the modified RNN using the token, the training comprising changing the modified RNN to a state associated with the at least a portion of the sequence of elements matching the token.

### Claim 31 (depends on 30)

The system of claim 30, wherein: the classifier is configured to identify the sequence of elements as matching a string.

### Claim 32 (depends on 30)

The system of claim 30, wherein: the sequence of elements is identical in composition to the elements in the string.

### Claim 33 (depends on 30)

The system of claim 30, wherein: modifying the RNN to comprises augmenting the RNN with one or more weights associated with the token.

### Claim 34 (depends on 30)

The system of claim 30, wherein: the classifier is trained using training data that is labeled to indicate the existence of one or more strings.

### Claim 35 (depends on 30)

The system of claim 30, wherein: the classifier is trained using training data that comprises key-value pairs.

### Claim 36 (depends on 30)

The system of claim 30, wherein: at least one of the first vocabulary or the second vocabulary includes strings from database records.

### Claim 37 (depends on 30)

The system of claim 30, wherein:
training the modified RNN comprises a forward phase and backward phase; and
the forward phase for training comprises sequentially inputting each element in a training sequence to the RNN.

### Claim 38

A system comprising: at least one processor; and
at least one memory containing instructions that, when executed by the at least one processor, causes the system to perform operations comprising:
obtaining a sequence of elements;
determining, using a classifier trained to identify matches to a token, that at least a portion of the sequence of elements matches the token; and
in response to the determination:
modifying a recurrent neural network (RNN) to use a second vocabulary, wherein the second vocabulary includes the token and the first vocabulary; and
training the modified RNN using the token, the training comprising reverting the modified RNN to a state associated with the at least a portion of the sequence of elements matching the token.

### Claim 39 (depends on 37)

The system of claim 37, wherein: training the modified RNN using the token comprises a forward phase and backward phase, wherein the backward phase for training includes one or more weights of the RNN, being updated using a backpropagation-through-time.

### Claim 40 (depends on 38)

The system of claim 38, wherein: the classifier identifies matching portions of a training example in parallel with the training of the RNN, the training data being updated for the interruption of the training to repeat the forward phase using a portion of the updated training example.
