Efficient processing of convolutional neural network layers using analog-memory-based hardware

An electronic circuit for performing computations of a trained convolutional neural network (CNN), the electronic circuit comprising:

• a crosspoint array;
• and
• an output circuit comprising one or more integrators;
• wherein performing the computations of the trained CNN comprises:
• the crosspoint array configured to be a convolution layer in the CNN by storing one or more convolution kernels of the convolution layer in one or more crosspoint devices of the crosspoint array, and to perform computations for the CNN via the crosspoint array, the electronic circuit further comprises an input circuitry comprising at least a support circuitry, a shared circuitry, and a row circuitry, the row circuitry including hardware components associated with each row wire and wherein the electronic circuit is configured to:
• transmit, by the input circuitry, to the crosspoint array, voltage pulses corresponding to a vector of input data of the convolution layer;
• output an electric current representative of performing a multiplication operation at each crosspoint device in the crosspoint array, the electric current based on a weight value stored by each crosspoint device and the voltage pulses from the input data;
• and
• pass the output electric current from crosspoint devices from multiple columns of the crosspoint array from the output circuit to corresponding integrators selected from a set of integrators, where each corresponding integrator corresponds to at least three columns of the multiple columns and is electrically switchable between the at least three columns of the multiple columns so as to aggregate current from one of the at least three columns of the at multiple columns during a single time step, wherein each corresponding integrator is selected by an output controller that maps the output electric current from the at least three columns to selected integrators and that switches which column of the at least three columns of the multiple columns each integrator in the set of integrators is connected to between time steps, wherein the corresponding integrators are capacitors, and wherein the output electric current from crosspoint devices from each of the multiple columns of the crosspoint array is passed to the corresponding integrators within a common time step.

An electronic circuit comprising:

• an array of resistive memory elements, the array providing a vector of current outputs equal to an analog vector-matrix-product between (i) a vector of voltage inputs to the array encoding a vector of analog input values and (ii) a matrix of analog resistive weights within the array;
• accumulation wires and circuits aggregating a current from a dedicated subset of the resistive memory elements;
• integration capacitors, each of the integration capacitors being electrically switchable so as to aggregate current from a plurality of accumulation wires with at least two of the integration capacitors being configured to aggregate current from corresponding sets of at least three accumulation wires, the plurality of accumulation wires selected using a mode signal, and the plurality of accumulation wires selected being changed between time steps;
• data-output circuitry to allow an integrated charge from a subset of the integration capacitors, accumulated over a plurality of integration steps, to be suitably converted and transmitted either as an analog duration or as a digital representation using binary digits;
• wherein the resistive memory elements are arranged so as to implement columns of weight kernels of a given layer of a convolutional neural network;
• wherein accumulation over the plurality of integration steps implements multiply-accumulate operations across multiple rows of said weight kernels, as input neuron excitations to the said layer of the convolutional neural network are presented one row at a time;
• wherein the integrated charge representing an output excitation is suitably converted and transmitted only after all rows of said weight kernels are fully integrated;
• and
• wherein the integrated charge on a plurality of capacitors representing a plurality of output excitations is suitably converted and a suitable pooled result comprising at least one of a maximum, a sum, or an average of the said plurality of output excitations is computed locally and then transmitted, only after all relevant weight kernels are fully integrated.