Neural network unit that performs concurrent LSTM cell calculations

A method for operating an apparatus having an output buffer, an array of N neural processing units (NPU), and a plurality of activation function units, wherein each of the plurality of activation function units is implemented in circuitry, the output buffer holds N words arranged as N/J output buffer word groups of J words each of the N words, J is an integer number greater than 3 and N is an integer number at least sixteen times J, the array of N NPUs are arranged as N/J NPU groups of J NPUs each of the N NPUs, each NPU group of the N/J NPU groups has an associated output buffer word group of the N/J output buffer word groups, each NPU of the N NPUs includes an accumulator having an output and an arithmetic unit that performs operations on inputs, which include the accumulator output, to generate a first result for accumulation into the accumulator, the plurality of activation function units selectively perform an activation function on the outputs of the accumulators to generate results for provision to the N output buffer words, the method comprising:

• for each NPU group of the N/J NPU groups of J NPUs:
• computing, by first, second, third and fourth of the J NPUs of the NPU group and at least one of the plurality of activation function units, an input gate, a forget gate, an output gate and a candidate cell state of a Long Short Term Memory (LSTM) cell, respectively, for writing to respective first, second, third and fourth words of the associated output buffer word group.

A computer program product encoded in at least one non-transitory computer usable medium for use with a computing device, the computer program product comprising:

• computer usable program code embodied in said medium, for specifying an apparatus, the computer usable program code comprising:
• first program code for specifying an output buffer that holds N words arranged as N/J output buffer word groups of J words each of the N words, J is an integer number greater than 3 and N is an integer number at least sixteen times J;
• second program code for specifying an array of N neural processing units (NPU) arranged as N/J mutually exclusive NPU groups of J NPUs each of the N NPUs, each NPU group of the N/J NPU groups has an associated output buffer word group of the N/J output buffer word groups, each NPU of the N NPUs includes an accumulator having an output and an arithmetic unit that performs operations on inputs, which include the accumulator output, to generate a first result for accumulation into the accumulator;
• third program code for specifying a plurality of activation function units that selectively perform an activation function on the outputs of the accumulators to generate results for provision to the N output buffer words, wherein each of the plurality of activation function units is implemented in circuitry;
• and
• for each NPU group of the N/J NPU groups of J NPUs:
• first, second, third and fourth of the J NPUs of the NPU group and at least one of the plurality of activation function units compute an input gate, a forget gate, an output gate and a candidate cell state of a Long Short Term Memory (LSTM) cell, respectively, for writing to respective first, second, third and fourth words of the associated output buffer word group.