Abstraction layers for scalable distributed machine learning

A system comprising:

• a system memory to store a set of trainable machine learning parameters and a library to facilitate data transmission during distributed training of a neural network;
• a fabric interface to enable transmission and receipt of data associated with the set of trainable machine learning parameters;
• a first set of general-purpose processor cores to execute instructions provided by the library, the instructions to control a data transmission library;
• and
• a general-purpose graphics processor to perform compute operations associated with machine learning framework workflow, the compute operations to generate gradient data for the trainable machine learning parameters, wherein the first set of general-purpose processor cores are to control the data transmission library to send and receive training data via the fabric interface during the machine learning framework workflow.

An apparatus comprising:

• a general-purpose graphics processor to:
• perform compute operations associated with a machine learning framework workflow;
• and
• generate, as part of the machine learning framework workflow, gradient data for trainable machine learning parameters during distributed training of a neural network;
• wherein a first set of general-purpose processor cores are to control a data transmission library to send and receive training data for the machine learning framework workflow via a fabric interface during the machine learning framework workflow.

A method comprising:

• performing, by a general-purpose graphics processor, compute operations associated with a machine learning framework workflow;
• and
• generating, by the general-purposes graphics processor as part of the machine learning framework workflow, gradient data for trainable machine learning parameters during distributed training of a neural network;
• wherein a first set of general-purpose processor cores are to control a data transmission library to send and receive training data for the machine learning framework workflow via a fabric interface during the machine learning framework workflow.

A non-transitory machine-readable storage medium having stored thereon executable computer program instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

• performing, by a general-purpose graphics processor of the one or more processors, compute operations associated with a machine learning framework workflow;
• and
• generating, by the general-purposes graphics processor as part of the machine learning framework workflow, gradient data for trainable machine learning parameters during distributed training of a neural network;
• wherein a first set of general-purpose processor cores are to control a data transmission library to send and receive training data for the machine learning framework workflow via a fabric interface during the machine learning framework workflow.