Deep learning for characterizing unseen categories
AssigneeUniversity of Florida Research Foundation, Inc.
InventorsDapeng Oliver Wu, Pratik Brahma
Life of this patent
Estimating the term…
Abstract
Systems and methods for a computer-based visual recognition based upon a spatially forked deep learning architecture, including unification of deep learning and reasoning. A method can include a primary learner with an adjacent structured memory bank that can not only predict the output from a given input, but also relate the input to all past memorized instances and help in creative understanding.
Claims20
A computer-based method of visual recognition based at least in part on a spatially forked deep learning architecture, the method comprising:
- training a primary learner with training data, the primary learner comprising:
- an input layer configured to receive an input;
- a plurality of hidden layers comprising a plurality of nodes and edges;
- and
- an output layer configured to output an output class of the input;
- sampling intermediate representations of the training data from at least one particular layer of the plurality of hidden layers of the primary learner;
- generating a representation memory bank associated with a comparator network, the representation memory bank comprising classes based at least in part on the sampled intermediate representations;
- training the comparator network to output a ternary conditional variable indicating a comparative relationship between a first comparator input and a second comparator input, wherein the comparator network is trained using an adjacency matrix representative of relationships between pairs of tensors corresponding to the classes from the representation memory bank;
- providing an unknown test input to the comparator network;
- comparing the unknown test input with the representation memory bank using the comparator network;
- generating an output of the comparator network based at least in part on the comparison, the output characterizing the unknown test input symbolically with respect to a known output class;
- and
- determining whether the unknown test input belongs to a known output class based at least in part on an output of the comparator network.
A deep learning apparatus in electronic communication with an input device configured to receive an input and an output device configured to receive an output from the deep learning apparatus, the deep learning apparatus comprising a computer readable medium comprising instructions that, when executed, cause at least one processor to:
- train a primary learner with training data, the primary learner comprising:
- an input layer configured to receive the input;
- a plurality of hidden layers comprising a plurality of nodes and edges;
- and
- an output layer configured to output an output class of the input;
- sample intermediate representations of the training data from at least one particular layer of the plurality of hidden layers of the primary learner;
- generate a representation memory bank associated with a comparator network, the representation memory bank comprising classes based at least in part on the sampled intermediate representations;
- train the comparator network to output a ternary conditional variable indicating a comparative relationship between a first comparator input and a second comparator input, wherein the comparator network is trained using an adjacency matrix representative of relationships between pairs of tensors corresponding to the classes from the representation memory bank;
- receive a test input via electronic communication with the input device;
- compare the test input with the representation memory bank using the comparator network;
- generate an output of the comparator network based at least in part on the comparison, the output characterizing the test input symbolically with respect to a known output class;
- and
- determine whether the test input belongs to a known output class based at least in part on an output of the comparator network.
A device for computer-based visual recognition based at least in part on a spatially forked deep learning architecture, the device comprising:
- a non-transitory computer readable medium comprising instructions that, when executed, cause at least one processor to:
- train a primary learner with training data, the primary leaner comprising:
- an input layer configured to receive the input;
- a plurality of hidden layers comprising a plurality of nodes and edges;
- and
- an output layer configured to output an output class of the image;
- sample intermediate representations of the training data from at least one particular layer of the plurality of hidden layers of the primary learner;
- generate a representation memory bank associated with a comparator network, the representation memory bank comprising classes based at least in part on the sampled intermediate representations;
- train the comparator network to output a ternary conditional variable indicating a comparative relationship between a first comparator input and a second comparator input, wherein the comparator network is trained using an adjacency matrix representative of relationships between pairs of tensors corresponding to the classes from the representation memory bank;
- receive a test input provided by a user;
- compare the test input with the representation memory bank using the comparator network;
- generate an output of the comparator network based at least in part on the comparison, the output characterizing the test input symbolically with respect to a known output class;
- and
- determine whether the test input belongs to a known output class based at least in part on an output of the comparator network.
Dependencies are inferred from the claim text.