Method and apparatus for adaptive learning in neural networks

A method of controlling a computer to adjust a neural network, said computer including an artificial neural network having multiple layers including an input layer and an output layer, each layer having a plurality of neurons, each neuron in a first layer is connected to each neuron in a second layer by a weight, said weights organized into a weight vector, comprising the steps of:

• assigning in said computer a first memory location for storing a first learning rate, a second memory location for storing a momentum factor, a memory block for storing said weight vector, a third memory location for storing a second learning rate;
• selecting said first learning rate, momentum factor, and said weight vector;
• storing said first learning rate, said momentum factor, and said weight vector into their respective memory locations assigned in said assigning step;
• utilizing an error function and a set of test patterns to compute a total error for said neural network using said weight vector;
• designating said first learning rate to said second learning rate by storing said first learning rate into said third memory location;
• using a search technique to adjust said first learning rate as a function of said total error by allowing said momentum factor to drift in a direction dependent upon said first learning rate and said second learning rate;
• reversing said direction if said momentum factor is equal to an upper limit or a lower limit;
• adapting said momentum factor based upon said second learning rate and said first learning rate;
• utilizing said first learning rate and said momentum factor to adjust said weight vector;
• meeting a predetermined convergence criterion;
• and
• updating said weight vector to said artificial neural network.

An improved neural network adjusted to match a known set of test patterns comprising:

• a neural network having multiple layers including an input layer and an output layer, each layer having a plurality of neurons, each of said plurality of neurons in a first layer connected to each of said plurality of neurons in a second layer by a weight, said weights organized into a weight vector;
• a memory storing a set of signals associated with said known set of test patterns, a first learning rate, a second learning rate, a momentum factor, and said weight vector;
• and
• processor means coupled to said neural network and said memory for comparing a set of signals from said output layer to said set of signals associated with said known set of test patterns to determine errors;
• said processor means if said errors exceed a predetermined error convergence criteria replacing in said memory said second learning rate with said first learning rate, readjusting said first learning rate and adjusting said momentum factor by a predetermined incremental value and multiplying said predetermined incremental value by a negative value if said first learning rate is increasing or if said momentum factor is equal to an upper limit or a lower limit, and readjusting said weight vector stored in said memory using the readjusted first learning rate and the adjusted momentum factor.