Learning machine synapse processor system apparatus

A method of emulating back-propagation learning on a neural network device comprising the steps of:

• a) initializing weight value registers and neuron output value registers;
• b) switching the device to a forward execution mode and inputting into at least one first layer neuron a data pattern;
• c) selected neurons each issuing a multiply instruction and a portion of said selected neurons specifying one of a plurality of adder trees each coupled to each one of said portion of said selected neurons for carrying out a summation operation;
• d) executing a first layer of said selected neurons for creating first neuron layer outputs and derivatives;
• e) feeding the first layer neuron outputs back into the device including reverse communicating the first layer neuron outputs and executing a second layer of neurons for creating second neuron layer outputs and derivatives;
• f) executing remaining layers of neurons in succession, each succeeding layer executing after a previously executed layer has fed its outputs back into the device;
• g) switching the device into a learning mode after all layers of neuron outputs and derivatives have been stored in neuron output and derivative arrays;
• and
• h) checking neuron outputs for convergence to specified values with a minimum specified error and terminating the learning mode if the minimum specified error is reached.

A method of emulating a Boltzmann machine on a neural network device by asynchronously executing individual neurons, comprising the steps of:

• a) clamping input and output neuron units to a selected encoder pattern including reverse communicating network input values and network output values to their respective neuron units;
• b) randomly selecting a hidden neuron unit for updating including issuing to the hidden neuron unit a multiply command and a command to select one of a plurality of coupled adder trees for carrying out a summation;
• c) determining an on-state probability of the selected hidden neuron unit;
• d) determining the selected hidden neuron unit's activation value;
• e) reverse communicating the activation value through the network;
• f) continuing steps b-e until the network reaches equilibrium as defined by an annealing schedule;
• g) selecting a remaining encoder pattern and, if any, repeating steps a-f;
• and
• h) storing performance statistics over all the encoder patterns run on the network which represents a probability that a pair of connected neurons were "on" simultaneously.

A method for performing matrix multiplication on a neural network device via parallel operation of the synapse processors comprising the steps of:

• a) loading a synapse processor array with matrix values of a first matrix;
• b) loading a second synapse processor array with first row matrix values of a second matrix by reverse communicating the first row matrix values through communicating adder trees;
• c) issuing a multiply instruction and initializing the communicating adder trees for a summation operation;
• d) calculating first row values of a result matrix by multiplying the matrix values of the first matrix by the first row matrix values of the second matrix followed by the summation operation;
• e) storing the values of the result matrix in a programmable processor controlling apparatus;
• f) loading next row matrix values, if any, of the second matrix by reverse communication through the communicating adder trees;
• g) if the loading of next row matrix values of step f) was performed, calculating next row values of a result matrix by multiplying the matrix values of the first matrix by the next row matrix values of the second matrix followed by the summation operation;
• h) if the calculation of next row values of step g) was performed, storing the next row values of the result matrix in the programmable processor controlling apparatus;
• and
• i) repeating steps f)-h) until a final row of the result matrix is calculated and stored.