Training a neural network using small training datasets
AssigneeArimo, LLC
InventorsChristopher T. Nguyen, Binh Han
Life of this patent
Estimating the term…
Abstract
Training datasets are determined for training neural networks. An input dataset comprising a plurality of samples is provided as training dataset to the neural network. Vector representations of samples of the input dataset are obtained from a hidden layer of the neural network. The samples are clustered using the vector representation. The samples are scored based on a metric that indicates the similarity of the sample to its cluster. A subset of samples is determined by excluding samples that have high similarity with their clusters. The subset of samples is labelled and used for training the neural network.
Claims20
A computer-implemented method comprising:
- training a neural network using a first training dataset, the neural network comprising a plurality of layers of nodes, the plurality of layers of nodes including a hidden layer of nodes;
- receiving an input dataset comprising a plurality of samples;
- executing the neural network to process samples of the input dataset;
- for each sample of the input dataset, receiving a vector representing the sample, the vector generated by nodes of the hidden layer of nodes;
- selecting a subset of samples from the input dataset based on distances between vectors representing the samples;
- for each sample of the subset, associating the sample with a label representing an expected result that should be output by the neural network for the sample;
- and
- providing the samples of the subset as a second training dataset for retraining the neural network.
A computer readable non-transitory storage medium, storing instructions for:
- training a neural network using a first training dataset, the neural network comprising a plurality of layers of nodes, the plurality of layers of nodes including a hidden layer of nodes;
- receiving an input dataset comprising a plurality of samples;
- executing the neural network to process samples of the input dataset;
- for each sample of the input dataset, receiving a vector representing the sample, the vector generated by nodes of the hidden layer of nodes;
- selecting a subset of samples from the input dataset based on distances between vectors representing the samples;
- for each sample of the subset, associating the sample with a label representing an expected result that should be output by the neural network for the sample;
- and
- providing the samples of the subset as a second training dataset for retraining the neural network.
A computer-implemented system comprising:
- a computer processor;
- and
- a computer readable non-transitory storage medium storing instructions thereon, the instructions when executed by a processor cause the processor to perform the steps of:
- training a neural network using a first training dataset, the neural network comprising a plurality of layers of nodes, the plurality of layers of nodes including a hidden layer of nodes;
- receiving an input dataset comprising a plurality of samples;
- executing the neural network to process samples of the input dataset;
- for each sample of the input dataset, receiving a vector representing the sample, the vector generated by nodes of the hidden layer of nodes;
- selecting a subset of samples from the input dataset based on distances between vectors representing the samples;
- for each sample of the subset, associating the sample with a label representing an expected result that should be output by the neural network for the sample;
- and
- providing the samples of the subset as a second training dataset for retraining the neural network.
Dependencies are inferred from the claim text.