Training of artificial neural networks

Apparatus for implementing an artificial neural network having a succession of neuron layers with interposed synaptic layers each having a respective set of N-bit fixed-point weights {w} for weighting signals propagated between its adjacent neuron layers in an iterative training cycle of signal propagation and weight-update calculation operations, the apparatus comprising:

• digital memory storing a plurality p of least-significant bits of each N-bit weight w of each synaptic layer;
• an analog multiply-accumulate unit for storing an n-bit portion of each weight w of the synaptic layer, the multiply-accumulate unit comprising an array of digital memory elements, each comprising n binary memory cells for storing respective bits of said n-bit portion of each weight, where 1≤n≤N and (p+n+m)=N where m≥0 corresponds to a defined number of most-significant zero bits in weights of the synaptic layer;
• and
• a digital processing unit operatively coupled to said digital memory and multiply-accumulate unit, the digital processing unit being adapted:
• in a signal propagation operation, to supply signals to be weighted by each synaptic layer to the multiply-accumulate unit to obtain accumulated weighted signals dependent on the stored n-bit portions of the weights;
• to perform a weight-update calculation operation to calculate updated weights for each synaptic layer in dependence on signals propagated by the neuron layers;
• and
• to control periodic reprogramming of the digital memory and the multiply-accumulate unit to store the updated weights.