US 2022/0188075 A1 · FLOATING POINT MULTIPLY-ACCUMULATE UNIT FOR DEEP LEARNING

Abstract

A FPMAC operation has two operands: an input operand and a weight operand. The operands may have a format of FP16, BF16, or INT8. Each operand is split into two portions. The two portions are stored in separate storage units. Then operands are transferred to register files of a PE, with each register file storing bits of an operand sequentially. The PE performs the FPMAC operation based on the operands. The PE may include an FPMAC unit configured to compute an individual partial sum of the PE. The PE may also include an FP adder to accumulate the individual partial sum with other data, such as an output from another PE or an output form another PE array. The FP adder may be fused with the FPMAC unit in a single circuit that can do speculative alignment and has separate critical paths for alignment and normalization.

Claims25

Dependent claims

An apparatus for a deep neural network (DNN) includes:

a concatenating module configured to generate an input operand of a floating-point multiply-accumulate (FPMAC) operation, the input operand including a plurality of bits, wherein a first portion of the input operand is stored in a first storage unit of a memory, and a second portion of the input operand is stored in a second storage unit of the memory;
and
a processing element comprising:
a register file configured to store the input operand by storing the plurality of bits in a sequence,
a multiplier configured to perform a series of multiplication operations with the plurality of bits based on the sequence, wherein each multiplication operation is with one of the plurality of bits, and
an accumulator configured to perform an accumulation operation with results of the series of multiplication operations and to generate an individual partial sum of the processing element.

3 more elements

One or more non-transitory computer-readable media storing instructions executable to perform operations, the operations comprising:

transferring a first portion of an input operand of a floating-point multiply-accumulate (FPMAC) operation and a second portion of the input operand from two separate storage units of a memory to an input register file in a processing element, bits in the input operand having a first sequence in the input register file;
transferring a first portion of a weight operand of the FPMAC operation and a second portion of the weight operand from two other separate storage units of the memory to a weight register file in the processing element, bits in the weight operand having a second sequence in the input register file;
feeding an FPMAC unit of the processing element with the bits in the input operand and the bits in the weight operand based on the first sequence and the second sequence;
and
performing, by the FPMAC unit, the FPMAC operation based on the input operand and the weight operand to generate an individual partial sum of the processing element.

2 more elements

Dependencies are inferred from the claim text.

Floating point multiply-accumulate unit for deep learning

Life of this patent

Floating point multiply-accumulate unit for deep learning

Life of this patent

Abstract

Claims25