Configurable neural network integrated circuit

A neural network integrated circuit (IC) having a programmable architecture, comprising:

• a plurality of processing elements (PEs), including a plurality of PEs arranged in n layers, that generate output values at successive computation cycles, said computation cycles having n subcycles, one for each layer, each PE comprising:
• a data memory,
• an input weight memory,
• an output weight memory,
• a processing circuit that computes an intermediate value that is a function of the contents of said data and input weight memories, the contents of the output weight memory and the PE's output values from a previous cycle, and
• an activation circuit that maps the intermediate value into said output value in accordance with a desired activation function;
• an output memory for storing said output values so that said stored output values are accessible to each PE's processing circuit at the next computation cycle;
• and
• a PE selection circuit that stores an execution subcycle for each PE and controls said output memory so that the PE's output values are only written into said output memory during their respective execution subcycles,
• the input and output weight memories being programmable so that the PEs can be interconnected to provide a desired neural network architecture.

A neural network processor, comprising:

• a neural network integrated circuit that includes a plurality of processing elements (PEs), including a plurality of PEs arranged in n layers, which generate output values at successive computation cycles, said computation cycle having n subcycles, one for each layer, and an output memory that stores the output values so that they are accessible to the PE's at the next computation cycle, each PE comprising:
• a data memory for storing the input data values,
• an input weight memory for storing input weights,
• an output weight memory for storing output weights, said input and output weights configuring the PEs to provide a desired neural network architecture, and
• a processing circuit that computes said output value as a function of the input data values and input weights, and the output weights and PEs' output values from a previous cycle;
• a microprocessor that writes input data into the PEs' data memories, and controls a start cycle to initiate the computation of said output values, and an output enable to read the output values;
• a timing circuit that receives a clock signal, initiates computation of the PEs in response to said start cycle, counts a fixed number of clock cycles per subcycle, increments a count at the end of each subcycle and generates an interrupt at the end of said computation cycle;
• and
• a PE selection circuit that stores an execution subcycle for each PE and controls said output memory so that the PE's output values are only written into said output memory when their respective execution subcycles are the same as the said count.