Guided augmention of data sets for machine learning models
AssigneeOracle International Corporation
InventorsAriel Gedaliah Kobren, Swetasudha Panda, Michael Louis Wick, Qinlan Shen, Jason Anthony Peck
Life of this patent
Estimating the term…
Abstract
Techniques are disclosed for augmenting data sets used for training machine learning models and for generating predictions by trained machine learning models. These techniques may increase a number and diversity of examples within an initial training dataset of sentences by extracting a subset of words from the existing training dataset of sentences. The techniques may conserve scarce sample data in few-shot situations by training a data generation model using general data obtained from a general data source.
Claims20
One or more non-transitory computer-readable media storing program instructions that, when executed by one or more hardware processors, cause performance of operations comprising:
- constructing a set of training inputs for training a synthetic data generation model, wherein the constructing the set of training inputs comprises:
- extracting, from a first training data set, a plurality of pairs of sentences that meet a similarity criterion, individual pairs of the plurality of pairs including a first sentence and a second sentence;
- for the individual pairs of the plurality of pairs:
- extracting a first subset of words from the first sentence, the first subset excluding one or more words included in the first sentence;
- and
- generating a first training instance from the set of training inputs, the first training instance comprising:
- (a) a model input including the second sentence and the subset of words from the first sentence, and (b) a model output including the first sentence.
A method comprising:
- constructing a set of training inputs for training a synthetic data generation model, wherein constructing the set of training inputs comprises:
- extracting, from a first training data set, a plurality of pairs of sentences that meet a similarity criterion, individual pairs of the plurality of pairs including a first sentence and a second sentence;
- for the individual pairs of the plurality of pairs:
- extracting a first subset of words from the first sentence, the first subset excluding one or more words included in the first sentence;
- and
- generating a first training instance from the set of training inputs, the first training instance comprising:
- (a) a model input including the second sentence and the subset of words from the first sentence, and (b) a model output including the first sentence.
A system comprising:
- a processor;
- and
- a computer-readable data storage device storing program instructions that, when executed by the processor, control the system to perform operations comprising:
- constructing a set of training inputs for training a synthetic data generation model, wherein the constructing the set of training inputs comprises:
- extracting, from a first training data set, a plurality of pairs of sentences that meet a similarity criterion, individual pairs of the plurality of pairs including a first sentence and a second sentence;
- for the individual pairs of the plurality of pairs:
- extracting a first subset of words from the first sentence, the first subset excluding one or more words included in the first sentence;
- and
- generating a first training instance from the set of training inputs, the first training instance comprising:
- (a) a model input including the second sentence and the subset of words from the first sentence, and (b) a model output including the first sentence.
Dependencies are inferred from the claim text.