NGraph-based GPU backend distributed training method and system

A non-transitory computer-readable storage medium, storing a computer program, wherein the computer program, upon execution by a processor, is configured to cause the processor to:

• receive a training request, and obtain corresponding training data;
• obtain a Collective multi-GPU Communication Library (CCL) file by means of a system path of the CCL file linked to an nGraph framework;
• obtain a communication operation function, and configure corresponding communication operation interfaces for operations, to establish a mapping between communication operation of the nGraph-based GPU backend and corresponding communication operation in the CCL;
• perform parameter parsing on the communication operation function;
• establish a function call relationship between parameters obtained by the parsing and a corresponding operation of the CCL operation, to obtain a CCL communication interface;
• invoke a CCL communication interface configuration according to the training request to obtain a training model, wherein the CCL communication interface is a CCL file-based communication operation interface located at a GPU backend of the nGraph framework;
• determine a distributed training type of the training model according to the training request, wherein the distributed training type comprises a multi-machine distributed training or a single-machine distributed training;
• in response to the distributed training type of the training model comprising the multi-machine distributed training, perform message passing interface (MPI) initialization and CCL initialization;
• and
• in response to the distributed training type of the training model comprising the single-machine distributed training, perform CCL initialization;
• and
• perform GPU backend training on the training data using the training model.

An electronic device, comprising:

• a memory and a processor, wherein the memory stores a computer program, and the processor, upon execution of the computer program, is configured to:
• receive a training request, and obtain corresponding training data;
• obtain a Collective multi-GPU Communication Library (CCL) file by means of a system path of the CCL file linked to an nGraph framework;
• obtain a communication operation function, and configure corresponding communication operation interfaces for operations, to establish a mapping between communication operation of the nGraph-based GPU backend and corresponding communication operation in the CCL;
• perform parameter parsing on the communication operation function;
• establish a function call relationship between parameters obtained by the parsing and a corresponding operation of the CCL operation, to obtain a CCL communication interface;
• invoke a CCL communication interface configuration according to the training request to obtain a training model, wherein the CCL communication interface is a CCL file-based communication operation interface located at a GPU backend of the nGraph framework;
• determine a distributed training type of the training model according to the training request, wherein the distributed training type comprises a multi-machine distributed training or a single-machine distributed training;
• in response to the distributed training type of the training model comprising the multi-machine distributed training, perform message passing interface (MPI) initialization and CCL initialization;
• and
• in response to the distributed training type of the training model comprising the single-machine distributed training, perform CCL initialization;
• and
• perform GPU backend training on the training data using the training model.