Delta-sigma modulation neurons for high-precision training of memristive synapses in deep neural networks

An artificial neural network (NN) circuit comprising:

• a plurality of interconnected neural network elements, each neural network element comprising:
• a neuron circuit comprising:
• a first delta-sigma modulator configured to receive input voltage signal and generate a modulated representation of the input voltage signal;
• and an averaging filter, wherein a transfer function of the neuron circuit serves as an activation function;
• and
• at least one synapse device comprising a memristor connected to an output of said neuron circuit, wherein a conductance of the memristor represents an adjustable synaptic weight of said at least one synapse device;
• and
• a memristor driver circuit comprising:
• at least one second delta-sigma modulator configured to receive at least one stored weighted sum and at least one desired value form a training dataset;
• at least one integrator module adapted to implement an averaging filter on output of said second delta-sigma modulator;
• at least one subtractor configured to compute error signal;
• and
• wherein said neural network (NN) is trained by an iterative process comprising:
• a read phase comprising:
• (i) inputting the at least input voltage signal into said neuron circuits;
• and
• (ii) calculating and storing a respective weighted sum of outputs of said neuron circuits, based on said synaptic weights;
• and
• an update phase comprising:
• (i) feeding the at least one stored weighted sum and the at least one desired value from the training dataset to the second delta-sigma modulator to generate two modulated outputs;
• (ii) calculating a cost function by subtracting the two modulated outputs using said subtractor;
• and
• (iii) adjusting said synaptic weights to minimize the cost function based on a gradient descent algorithm.