Recurrent neural networks teaching system
AssigneeAllen-Bradley Company, Inc.
InventorsMichael L. Gasperi, Wesley Davis
Life of this patent
Estimating the term…
Abstract
A teaching method for a recurrent neural network having hidden, output and input neurons calculates weighting errors over a limited number of propagations of the network. This process permits the use of conventional teaching sets, such as are used with feedforward networks, to be used with recurrent networks. The teaching outputs are substituted for the computed activations of the output neurons in the forward propagation and error correction stages. Back propagated error from the last propagation is assumed to be zero for the hidden neurons. A method of reducing drift of the network with respect to a modeled process is also described and a forced cycling method to eliminate the time lag between network input and output.
Claims7
A training apparatus for determining proper weights for recurrent output and recurrent hidden neurons in a recurrent neural network as derived from a teaching set of sequential teaching inputs and teaching outputs, the training apparatus comprising:
- a predetermined number of layers including a first layer and a last layer,
- each layer having layer inputs and layer outputs and layer error inputs and layer error outputs,
- the layer outputs of a given layer connected to the layer inputs of any immediately succeeding layer,
- the layer error outputs of any given layer connected to the layer error inputs of any immediately preceding layer,
- each of the layers including:
- a plurality of input neurons receiving teaching inputs and producing neuron outputs equal to those teaching inputs,
- a number of output neurons and hidden neurons, equal in number to the number of recurrent output neurons and recurrent hidden neurons in the recurrent neural network and having neuron inputs and neuron outputs and neuron error inputs and neuron error outputs,
- each output and hidden neuron having an associated set of weights independent of the other weights of that layer but identical to the weights of a corresponding neuron of the preceding and succeeding layer,
- wherein the output and hidden neurons generate neuron outputs as a function of neuron inputs multiplied by the weights,
- wherein the hidden neurons generate error outputs as a function of neuron error inputs multiplied by the weights and of the neuron output for that neuron,
- wherein the neuron inputs of the hidden and output neurons are connected to the layer inputs and the neuron outputs of the hidden and input neurons are connected to the layer outputs,
- wherein the neuron error inputs of the hidden neurons are connected to the layer error inputs and the neuron error outputs of the hidden and output neurons are connected to the layer error outputs;
- a weight corrector means for receiving final neuron error outputs from the layer preceding the first layer and changing the weights of the hidden and output neurons to produce corrected weights;
- and
- a communication means for transferring the corrected weights of the output and hidden neurons to corresponding recurrent output and recurrent hidden neurons in the recurrent neural network.
Dependencies are inferred from the claim text.