Method and apparatus for processing using neural network with reduced calculation amount

A neural network system, comprising:

• a plurality of neural network circuits which are independently operating with respect to each other, each neural network circuit receives a plurality of input values and outputs one output value, and includes:
• calculation means for carrying out a synapse calculation for each input value and a corresponding synapse weight of each input value which are expressed by binary bit sequences, by using all bits of each input value and each one bit of the corresponding synapse weight sequentially specified from upper bits to obtain a partial synapse calculation result for said each one bit and adding each partial synapse calculation result for said each one bit with previous partial synapse calculation results for upper bits of said each one bit, and a summation calculation for sequentially summing synapse calculation results for the input values to obtain a summation value;
• synapse weight memory circuit for storing a plurality of synapse weights;
• memory read out circuit for selectively reading out a specified bit of a specified synapse weight from the synapse weight memory circuit to the calculation means;
• a nonlinear unit for applying a prescribed nonlinear processing to the summation value obtained by the calculation means so as to determine the output value;
• judging means for judging whether the summation value obtained by the calculation means reached to a saturation region of a transfer characteristic of the nonlinear unit;
• and
• control means for controlling the memory read out circuit and the calculation means to sequentially carry out the synapse calculation from upper bits of the corresponding synapse weight, and to stop the synapse calculation and the summation calculation whenever the judging means judges that the summation value obtained by the calculation means reached to the saturation region;
• and
• a common input value register for sequentially supplying said plurality of input values to each neural network circuit such that an identical one of said plurality of input values is supplied to all of said plurality of neural network circuits simultaneously.

A neural network circuit receiving a plurality of input values and outputting one output value, comprising:

• reference input value register means for storing a plurality of reference input values in correspondence to said plurality of input values;
• subtractor means for calculating a difference between each input value and a corresponding one of said plurality of reference input values to obtain an input difference for each input value;
• calculation means for carrying out a synapse calculation for each input difference and a corresponding synapse weight of each input value and a summation calculation for summing synapse calculation results to obtain a summation value;
• synapse weight memory circuit for storing a plurality of synapse weights;
• memory read out circuit for selectively reading out the corresponding synapse weight from the synapse weight memory circuit to the calculation means;
• a nonlinear unit for applying a prescribed nonlinear processing to the summation value obtained by the calculation means so as to determine the output value;
• coincidence detection means for detecting a coincidence of each input value and the corresponding one of said plurality of reference input values;
• reference output value register means for storing a reference output value corresponding to said plurality of reference input values;
• switch means for selectively outputting one of the output value obtained by the nonlinear unit and the reference output value stored in the reference output value register means;
• and
• control means for controlling the calculation means and the switch means according to the coincidence detected by the coincidence detection means such that the calculation means stops the synapse calculation and the summation calculation while the switch means selectively outputs the reference output value stored in the reference output value register means whenever the coincidence detection means detects the coincidence, whereas the calculation means carries out the synapse calculation and the summation calculation while the switch means selectively outputs the output value obtained by the nonlinear unit otherwise.

A processing scheme using a neural network circuit receiving a plurality of input values and outputting one output value, comprising the steps of:

• (a) carrying out a synapse calculation for each input value and a corresponding synapse weight of each input value which are expressed by binary bit sequences, by using a sequentially specified bit of the corresponding synapse weight, and a summation calculation for sequentially summing synapse calculation results for the input values to obtain a summation value;
• (b) applying a prescribed nonlinear processing to the summation value obtained at the step (a) so as to determine the output value;
• (c) judging whether the summation value obtained at the step (a) reached to a saturation region of a transfer characteristic of the prescribed nonlinear processing applied at the step (b);
• and
• (d) controlling the step (a) to sequentially carry out the synapse calculation from upper bits of the corresponding synapse weight, and to stop the synapse calculation and the summation calculation whenever the step (c) judges that the summation value obtained at the step (a) reached to the saturation region.

A processing scheme using a neural network system, comprising the steps of:

• (a) independently operating a plurality of neural network circuits with respect to each other, by operating each neural network circuit which receives a plurality of input values and outputs one output value by the steps of:
• (a1) carrying out a synapse calculation for each input value and a corresponding synapse weight of each input value which are expressed by binary bit sequences, by using all bits of each input value and each one bit of the corresponding synapse weight sequentially specified from upper bits to obtain a partial synapse calculation result for said each one bit and adding each partial synapse calculation result for said each one bit with previous partial synapse calculation results for upper bits of said each one bit, and a summation calculation for sequentially summing synapse calculation results for the input values to obtain a summation value;
• (a2) applying a prescribed nonlinear processing to the summation value obtained at the step (a1) so as to determine the output value;
• (a3) judging whether the summation value obtained at the step (a1) reached to a saturation region of a transfer characteristic of the prescribed nonlinear processing applied at the step (a2);
• and
• (a4) controlling the step (a1) to sequentially carry out the synapse calculation from upper bits of the corresponding synapse weight, and to stop the synapse calculation and the summation calculation whenever the step (a3) judges that the summation value obtained at the step (a1) reached to the saturation region;
• and (b) sequentially supplying said plurality of input values to each neural network circuit such that an identical one of said plurality of input values is supplied to all of said plurality of neural network circuits simultaneously.

A processing scheme using a neural network circuit receiving a plurality of input values and outputting one output value, comprising the steps of:

• (a) storing in advance a plurality of reference input values in correspondence to said plurality of input values;
• (b) calculating a difference between each input value and a corresponding one of said plurality of reference input values to obtain an input difference for each input value;
• (c) carrying out a synapse calculation for each input difference and a corresponding synapse weight of each input value and a summation calculation for summing synapse calculation results to obtain a summation value;
• (d) applying a prescribed nonlinear processing to the summation value obtained at the step (c) so as to determine the output value;
• (e) detecting a coincidence of each input value and the corresponding one of said plurality of reference input values;
• (f) storing in advance a reference output value corresponding to said plurality of reference input values;
• (g) selectively outputting one of the output value obtained at the step (d) and the reference output value stored at the step (f);
• and
• (h) controlling the steps (c) and (g) according to the coincidence detected at the step (e) such that the step (c) stops the synapse calculation and the summation calculation while the step (g) selectively outputs the reference output value stored at the step (f) whenever the step (e) detects the coincidence, whereas the step (c) carries out the synapse calculation and the summation calculation while the step (g) selectively outputs the output value obtained at the step (d) otherwise.