Learning method for multilayer perceptron neural network with N-bit data representation

A learning method for a multilayer perceptron neural network with N-bit data representation, N being an integer, said method comprising the steps of:

• (i) setting an early weight for avoiding premature saturation of learning;
• (ii) presenting N-bit digital training pattern signals to a multilayer perceptron after performing step (i);
• (iii) generating weighted sum signals with 2N-bit data precision in a forward calculation process of a multilayer perceptron;
• (iv) converting the results of said 2N-bit weighted sum from step (ii) into N-bit data for sigmoidal nonlinear transformation;
• (v) determining whether an error signal of an output layer is generated;
• (vi) generating said error signal for said output layer if said error signal is generated in step (v);
• (vii) generating weighted sum signals with 2N-bit data precision in a back propagation of error signals in said multilayer perceptron;
• (viii) converting the weighted sum signals from step (vii) into N-bit data signals;
• (ix) changing weight signals according to said error signals of steps (vi) and (viii) to represent learning for said training pattern;
• (x) determining whether the weight signals from step (ix) reach the maximum value of weight signals in each layer;
• and
• (xi) expanding the range of weight signals being utilized to generate error signals if a predetermined portion of said weight signals reach said maximum value of the weight signals of said respective layers in step (x) and then presenting a next resultant training pattern to repeat step (ii) and the steps following thereafter.

In an improved method of training an N-bit multilayer perceptron neural network where N is an integer representing the number of bits in digital network input pattern signals, node state signals, weighting signals and error signals, the improvement comprising during a network training process:

• generating enhanced precision weighted sum signals having more than N-bit precision while deriving enhanced precision error signals in response to input pattern signals;
• and
• converting the resulting enhanced precision error signals back to N-bit precision for use in modifying the weight signals of the network during the training process;
• wherein the maximum value of a weighted sum converted into N-bit precision is set to a value in a saturated region of a sigmoidal transformation when the result of the weighted sum having enhanced data precision in the forward calculation of the multilayer perceptron is converted into one with N-bit precision for a sigmoidal nonlinear transformation of the weighted sum to generate the node state signal.

In an improved method of training an N-bit multilayer perceptron neural network where N is an integer representing the number of bits in digital network input pattern signals, node state signals, weighting signals and error signals, the improvement comprising during a network training process:

• generating enhanced precision weighted sum signals having more than N-bit precision while deriving enhanced precision error signals in response to input pattern signals;
• and
• converting the resulting enhanced precision error signals back to N-bit precision for use in modifying the weight signals of the network during the training process;
• wherein the maximum value of back propagated signal converted to N-bit precision is set to a value relatively smaller than that represented with 2N-bit precision when the enhanced precision back propagated weighted sum is represented with N-bit data between weight and error signals of an upper layer are converted into one with N-bit precision for updating a weight signal in proportion to the back propagated error signal.

In an improved method of training an N-bit multilayer perceptron neural network where N is an integer representing the number of bits in digital network input pattern signals, node state signals, weighting signals and error signals, the improvement comprising during a network training process:

• generating enhanced precision weighted sum signals having more than N-bit precision while deriving enhanced precision error signals in response to input pattern signals;
• and
• converting the resulting enhanced precision error signals back to N-bit precision for use in modifying the weight signals of the network during the training process;
• wherein the maximum value weight signals which can be represented with N-bit precision in the initial stage of learning is expanded if a predetermined ratio of the weights becomes the maximum value during the learning progress.