Systems and methods for enhanced matrix operations

A hardware accelerator comprising:

• a hardware compute unit comprising:
• a result register configured to store an accumulated value in a first floating-point format;
• a matrix multiplication unit configured to execute matrix multiplication operations in a second floating-point format, the second floating-point format having a different exponent range in comparison to the first floating-point format;
• wherein the hardware accelerator is configured to:
• receive a plurality of input tensors representing values converted from the first floating-point format to the second floating-point format;
• generate, via the hardware compute unit, a result tensor by:
• executing, via the matrix multiplication unit, a matrix operation using the plurality of input tensors in the second floating-point format;
• and
• accumulating intermediate results of the matrix operation in the result register within the hardware compute unit using the first floating-point format.

A system comprising:

• a host device configured to convert a plurality of input tensors from a first floating-point format to a second floating-point format, the second floating-point format having a different exponent range in comparison to the first floating-point format;
• a hardware accelerator configured to generate a result tensor by:
• executing a matrix operation using the plurality of input tensors in the second floating-point format;
• and
• accumulating intermediate results of the matrix operation in a result register using the first floating-point format.