Optimization of deep learning algorithms for large digital data processing using evolutionary neural networks
AssigneeVmc Mar Com Inc.
InventorMohammadreza NEHZATI
Life of this patent
Estimating the term…
Abstract
Embodiments of the present disclosure relate to neural networks for processing large digital datasets. Neural networks comprise both a convolutional neural network (CNN) and a recurrent neural network (RNN). The neural networks are optimized by applying genetic algorithms. Embedding vectors are processed by both the CNN and the RNN to produce a merged output.
Claims20
A method comprising:
- initializing a first plurality of neural networks;
- determining performance metrics, each of the performance metrics corresponding to a respective neural network of the first plurality of neural networks,
- wherein each of the performance metrics is based on accuracy of the respective neural network in evaluating training data;
- selecting a subset of the first plurality of neural networks based on the performance metrics;
- selecting a first neural network from the subset and a second neural network from the subset,
- wherein the first neural network comprises a first convolutional neural network (CNN) and a first recurrent neural network (RNN), and
- wherein the second neural network comprises a second CNN and a second RNN;
- forming a third neural network, wherein forming the third neural network comprises (a) selecting weights of the first CNN, selecting weights of the second CNN, and forming a third CNN of the third neural network at least in part from the selected weights of the first CNN and the second CNN and (b) selecting weights of the first RNN, selecting weights of the second RNN, and forming a third RNN of the third neural network at least in part from the selected weights of the first RNN and the second RNN;
- providing, to the third neural network, a plurality of embedding vectors generated from input data;
- processing, by the third CNN, a first subset of the embedding vectors, thereby producing a first output of the third CNN, wherein the third RNN does not process the first output of the third CNN;
- processing, by the third CNN, a second subset of the embedding vectors, thereby producing a second output of the third CNN;
- processing, by the third RNN, the second output of the third CNN, thereby producing an output of the third RNN;
- combining the first output of the third CNN with the output of the third RNN, thereby producing a merged output;
- and
- classifying the input data based on the merged output.
2-3. (canceled)
A system comprising:
- a processor;
- and
- memory storing instructions that, when executed by the processor, cause the processor to:
- initialize a first plurality of neural networks;
- determine performance metrics, each of the performance metrics corresponding to a respective neural network of the first plurality of neural networks,
- wherein each of the performance metrics is based on accuracy of the respective neural network in evaluating training data;
- select a subset of the first plurality of neural networks based on the performance metrics;
- select a first neural network from the subset and a second neural network from the subset,
- wherein the first neural network comprises a first convolutional neural network (CNN) and a first recurrent neural network (RNN), and
- wherein the second neural network comprises a second CNN and a second RNN;
- form a third neural network, wherein forming the third neural network comprises (a) selecting weights of the first CNN, selecting weights of the second CNN, and forming a third CNN of the third neural network at least in part from the selected weights of the first CNN and the second CNN and (b) selecting weights of the first RNN, selecting weights of the second RNN, and forming a third RNN of the third neural network at least in part from the selected weights of the first RNN and the second RNN;
- provide, to the third neural network, a plurality of embedding vectors generated from input data;
- process, by the third CNN, a first subset of the embedding vectors, thereby producing a first output of the third CNN, wherein the third RNN does not process the first output of the third CNN;
- process, by the third CNN, a second subset of the embedding vectors, thereby producing a second output of the third CNN;
- process, by the third RNN, the second output of the third CNN, thereby producing an output of the third RNN;
- combine the first output of the third CNN with the output of the RNN, thereby producing a merged output;
- and
- classify the input data based on the merged output.
16-17. (canceled)
A non-transitory computer readable medium having instructions thereon, the instructions, when executed by a computer, causing the computer to perform operations comprising:
- initializing a first plurality of neural networks;
- determining performance metrics, each of the performance metrics corresponding to a respective neural network of the first plurality of neural networks,
- wherein each of the performance metrics is based on accuracy of the respective neural network in evaluating training data;
- selecting a subset of the first plurality of neural networks based on the performance metrics;
- selecting a first neural network from the subset and a second neural network from the subset,
- wherein the first neural network comprises a first convolutional neural network (CNN) and a first recurrent neural network (RNN), and
- wherein the second neural network comprises a second CNN and a second RNN;
- and
- forming a third neural network, wherein forming the third neural network comprises (a) selecting weights of the first CNN, selecting weights of the second CNN, and forming a third CNN of the third neural network at least in part from the selected weights of the first CNN and the second CNN and (b) selecting weights of the first RNN, selecting weights of the second RNN, and forming a third RNN of the third neural network at least in part from the selected weights of the first RNN and the second RNN;
- providing, to the third neural network, a plurality of embedding vectors generated from input data;
- processing, by the third CNN, a first subset of the embedding vectors, thereby producing a first output of the third CNN, wherein the third RNN does not process the first output of the third CNN;
- processing, by the third CNN, a second subset of the embedding vectors, thereby producing a second output of the third CNN;
- processing, by the third RNN, the second output of the third CNN, thereby producing an output of the third RNN;
- combining the first output of the third CNN with the output of the third RNN, thereby producing a merged output;
- and
- classifying the input data based on the merged output.
19-20. (canceled)
Dependencies are inferred from the claim text.