Abstraction library to enable scalable distributed machine learning
AssigneeIntel Corporation
InventorsDhiraj D. Kalamkar, KARTHIKEYAN VAIDYANATHAN, SRINIVAS SRIDHARAN, DIPANKAR DAS
Life of this patent
Estimating the term…
Abstract
One embodiment provides for a non-transitory machine readable medium storing instructions which, when executed by one or more processors, cause the one or more processors to perform operations comprising providing an interface to define a neural network using machine-learning domain specific terminology, wherein the interface enables selection of a neural network topology and abstracts low-level communication details of distributed training of the neural network.
Claims20
A non-transitory machine readable medium storing instructions which, when executed by one or more processors, cause the one or more processors to perform operations comprising:
- providing an interface to define a neural network using machine-learning domain specific terminology;
- and
- wherein the interface enables selection of a neural network topology and abstracts low-level communication details of distributed training of the neural network.
A method provided by a library stored on a data processing system, the method comprising:
- providing an interface to define a neural network using machine-learning domain specific terminology, wherein the interface enables selection of a neural network topology and abstracts low-level communication details of distributed training of the neural network;
- receiving a definition of the neural network, the definition including a number and type of layers of the neural network and a type of parallelism to apply for the neural network during the distributed training of the neural network;
- determining a communication pattern to apply during the distributed training of the neural network based on the type of parallelism to apply for the neural network;
- and
- providing instructions to a communication module to enable the communication pattern.
A system to compute and distribute data for distributed training of a neural network, the system comprising:
- system memory to store a set of trainable machine learning parameters and a library to facilitate data transmission during distributed training of the neural network, the library to provide an interface to define a neural network using machine-learning domain specific terminology, wherein the interface enables selection of a neural network topology and abstract low-level communication details of distributed training of the neural network;
- a fabric interface to enable transmission and receipt of data associated with the set of trainable machine learning parameters;
- a first set of general-purpose processor cores to execute instructions provided by the library, the instructions to control a data transmission library;
- and
- a general-purpose graphics processor to perform compute operations associated with a machine learning framework workflow to generate gradient data for the trainable machine learning parameters, wherein the first set of general-purpose processor cores are to control the data transmission library to send and receive training data via the fabric interface during the machine learning framework workflow.
Dependencies are inferred from the claim text.