Neural network accelerators resilient to conductance drift

A method comprising:

• applying a correction factor to at least one neuron input in a neural network having a plurality of neurons and a plurality of resistive processing unit (RPU) weights, wherein each neuron has an activation function and the correction factor is based on a drift coefficient of the plurality of RPU weights and a time that has elapsed since the RPU weights were programmed;
• applying an amplification factor to activation function outputs of the plurality of neurons in the neural network, the amplification factor selected independent of the drift coefficient of the plurality of RPU weights and independent of the time that has elapsed since the RPU weights were programmed;
• and
• performing a calculation with the neural network using the amplified activation function outputs of the plurality of neurons.

A system comprising:

• a neural network that includes a plurality of neurons and a plurality of resistive processing unit (RPU) weights, wherein each of the plurality of neurons has an activation function and the neural network has zero bias neurons;
• and
• a processing unit configured to apply an arbitrary amplification factor to activation function outputs of the one or more neurons in the neural network, wherein the arbitrary amplification factor is based on a dynamic range of components in the neural network and compensates for conductance drift in values of the RPU weights;
• and
• wherein the neural network is configured to perform a calculation using the amplified activation function outputs of the one or more neurons.

A system comprising:

• a neural network that includes a plurality of neurons and a plurality of resistive processing unit (RPU) weights, wherein each of the plurality of neurons has an activation function;
• and
• a processing unit configured to:
• apply a correction factor to at least one neuron input in the neural network, wherein the correction factor is based on a drift coefficient of the plurality of RPU weights and a time that has elapsed since the RPU weights were programmed;
• and
• apply an amplification factor to activation function outputs of the plurality of neurons in the neural network, wherein the amplification factor selected independent of the drift coefficient of the plurality of RPU weights and independent of the time that has elapsed since the RPU weights were programmed;
• and
• wherein the neural network is configured to perform a calculation using the amplified activation function outputs of the one or more neurons.

A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed by a processor, causes the processor to:

• applying an arbitrary amplification factor to activation function outputs of one or more neurons in a neural network;
• wherein the neural network has zero bias neurons and includes a plurality of resistive processing unit (RPU) weights and each neuron has an activation function;
• wherein the arbitrary amplification factor is based on a dynamic range of components in the neural network and compensates for conductance drift in values of the RPU weights;
• and
• wherein the neural network is configured to perform a calculation using the amplified activation function outputs of the one or more neurons.