Hardware accelerated machine learning

An apparatus comprising:

• an instruction decoder to decode a matrix multiplication instruction;
• a local memory comprising a plurality of static random-access memory (SRAM) banks to store at least a portion of a first input tensor and a second input tensor, each of the first and second input tensors comprising a multidimensional array of real numbers;
• circuitry to permute a first plurality of the real numbers associated with the first input tensor in accordance with a permutation pattern included with the matrix multiplication instruction to generate a permuted first plurality of real numbers;
• a hardware accelerator to multiply the first input tensor and the second input tensor to produce an output tensor, the hardware accelerator comprising:
• a plurality of operational units to perform a plurality of parallel multiply-accumulate (MAC) operations in accordance with the matrix multiplication instruction, each operational unit comprising:
• a multiplier to multiply a first real number of the permuted first plurality of real numbers and a second real number of a second plurality of real numbers associated with the second input tensor to generate a product, and
• an accumulator to add the product to an accumulation value to generate a result value, the first real number and the second real number each having a first bit width and the accumulation value having a second bit width at least twice the first bit width.