Abstraction layers for scalable distributed machine learning

A system to compute and distribute data for distributed training of a neural network, the system comprising:

• a system memory to store a set of trainable machine learning parameters and a library to facilitate data transmission during distributed training of the neural network;
• a fabric interface to enable transmission and receipt of data associated with the set of trainable machine learning parameters;
• a first set of general-purpose processor cores to execute instructions provided by the library, the instructions to control a data transmission library;
• and
• a general-purpose graphics processor to perform compute operations associated with machine learning framework workflow to generate gradient data for the trainable machine learning parameters, wherein the first set of general-purpose processor cores are to control the data transmission library to send and receive training data via the fabric interface during the machine learning framework workflow.