Neural networks with subdomain training
AssigneePassiveLogic, Inc.
InventorsTroy Aaron Harvey, Jeremy David Fillingim
Life of this patent
Estimating the term…
Abstract
Heterogenous neural networks are disclosed that have activation functions that hold multi-variable equations. These variables can be passed from one neuron to another. The neurons may be laid out in a topologically similar fashion to a physical system that the heterogenous neural network is modeling. A neural network may have inputs of more than one type. Only a portion of the inputs (a subdomain) may be optimized In such an instance, the neural network may run forward, backpropagate to all inputs, and then perform optimization only on those inputs which will be optimized.
Claims20
A system associated with a physical process, comprising:
- a processor;
- a memory storing a trained neural network and instructions executable by the processor;
- the trained neural network comprising:
- a plurality of neurons arranged in layers and interconnected by signal-transmission connections forming a transmissive signal path extending between the plurality of neurons across multiple layers;
- a plurality of local input connections, each local input connection coupled to a respective neuron and configured to supply a signal to that neuron without providing the signal to any other neuron through the signal-transmission connections;
- wherein:
- signals provided through the signal-transmission connections propagate between the plurality of neurons and contribute to activation of downstream neurons;
- and
- signals provided through the plurality of local input connections affect activation of the respective coupled neuron without propagating to other neurons;
- and
- wherein the plurality of local input connections are excluded from backpropagation of gradients, and
- wherein the trained neural network produces, after modification of the signals propagating through the transmissive signal path, an output signal structured for use in operating at least one actuator of the physical process.
A computer-implemented method associated with a physical process, comprising:
- executing, by at least one processor, a trained neural network comprising:
- a plurality of neurons arranged in layers and interconnected by signal-transmission connections forming a transmissive signal path extending between the plurality of neurons across multiple layers, each signal-transmission connection connecting a source neuron to a different destination neuron;
- a plurality of local input connections, each local input connection coupled to a respective neuron and configured to supply a signal to that neuron without providing the signal to any other neuron through the signal-transmission connections;
- wherein signals provided through the signal-transmission connections propagate between the plurality of neurons and contribute to activation of downstream neurons, and signals provided through the plurality of local input connections affect activation of the respective coupled neuron without propagating to other neurons;
- performing forward propagation through the trained neural network using both signals provided through the signal-transmission connections and signals provided through the plurality of local input connections;
- computing a cost function based at least in part on outputs of the trained neural network;
- computing gradients of the cost function with respect to signals propagating through the transmissive signal path while not computing gradients with respect to signals provided through the plurality of local input connections;
- modifying only the signals propagating through the transmissive signal path based on the computed gradients while maintaining the signals provided through the plurality of local input connections;
- and
- producing, after modifying only the signals propagating through the transmissive signal path, an output signal structured for use in operating at least one actuator of the physical process.
A non-transitory computer-readable medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform operations associated with a physical process, the operations comprising:
- executing a trained neural network stored in memory, the trained neural network comprising:
- a plurality of neurons arranged in layers and interconnected by signal-transmission connections forming a transmissive signal path extending between the plurality of neurons across multiple layers;
- and
- a plurality of local input connections. each local input connection coupled to a respective neuron and configured to supply a signal to that neuron without providing the signal to any other neuron through the signal-transmission connections;
- wherein:
- signals provided through the signal-transmission connections propagate between the plurality of neurons and contribute to activation of downstream neurons;
- signals provided through the plurality of local input connections affect activation of the respective coupled neuron without propagating to other neurons, and the plurality of local input connections are excluded from backpropagation of gradients;
- modifying signals propagating through the transmissive signal path;
- and
- producing, using the trained neural network after modification of the signals propagating through the transmissive signal path, an output signal structured for use in operating at least one actuator of the physical process.
Dependencies are inferred from the claim text.