Machine learning sparse computation mechanism for arbitrary neural networks, arithmetic compute microarchitecture, and sparsity for training mechanism

A graphics processing unit, comprising:

• a sparsity management unit to manage sparsity operations, wherein the sparsity management unit comprises:
• a value check mechanism to detect unimportant values within input vectors, the unimportant values including zero operands and redundant operands, and skip operations for the unimportant values of the input vectors, and
• a scheduler to determine scheduling of computations based on scheduling important values and skipping unimportant values of input vectors that are detected by the value check mechanism;
• a block floating point (FP) management unit 3120 to support block FP operations;
• and
• a variable and mix precision compute unit to support variable and mix precision operations.

A method for training of data, comprising:

• obtaining a first sparse matrix encoded with compressed sparse row (CSR) and a second dense matrix;
• offloading the second dense matrix in a coalesced manner from memory to a shared local memory (SLM);
• determining a minimum number of workgroups to launch to minimize a number of redundant global memory loads to the SLM;
• selecting a work group size for each of the minimum number of workgroups;
• and
• launching the minimum number of workgroups for execution on a graphics processing unit (GPU), wherein the minimum number of workgroups is determined based on a total number of hardware threads supported by the GPU and a number of data elements associated with a hardware thread, wherein the redundant global memory loads are associated with redundant operands associated with an irregular neural network having an arbitrary connection across non-adjacent layers of the neural network.