Structure learning in convolutional neural networks
AssigneeMagic Leap, Inc.
InventorsAndrew Rabinovich, Vijay Badrinarayanan, Daniel DeTone, Srivignesh Rajendran, Douglas Bertram Lee, Tomasz Malisiewicz
Life of this patent
Estimating the term…
Abstract
The present disclosure provides an improved approach to implement structure learning of neural networks by exploiting correlations in the data/problem the networks aim to solve. A greedy approach is described that finds bottlenecks of information gain from the bottom convolutional layers all the way to the fully connected layers. Rather than simply making the architecture deeper, additional computation and capacitance is only added where it is required.
Claims33
A method implemented with a processor, comprising:
- creating a neural network;
- generating output from the neural network;
- identifying a low performing layer from the neural network, the low performing layer having a relatively lower performance than a performance of another layer in the neural network;
- inserting a new specialist layer at the low performing layer;
- and
- repeating the act of identifying and the act of inserting until a top of the neural network is reached.
A system, comprising:
- a processor;
- a memory for holding programmable code;
- and
- wherein the programmable code includes instructions for creating a neural network;
- generating output from the neural network;
- identifying a low performing layer from the neural network, the low performing layer having a relatively lower performance than a performance of another layer in the neural network;
- inserting a new specialist layer at the low performing layer;
- and repeating the act of identifying and the act of inserting until a top of the neural network is reached.
A computer program product embodied on a non-transitory computer readable medium, the non-transitory computer readable medium having stored thereon a sequence of instructions which, when executed by a processor causes the processor to execute a method comprising:
- creating a neural network;
- generating output from the neural network;
- identifying a low performing layer from the neural network, the low performing layer having a relatively lower performance than a performance of another layer in the neural network;
- inserting a new specialist layer at the low performing layer;
- and
- repeating the act of identifying and the act of inserting until a top of the neural network is reached.
Dependencies are inferred from the claim text.