# US 12,001,960 B2 · NGraph-based GPU backend distributed training method and system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,001,960 B2
- Document ID: US12001960B2
- Status: Granted
- Kind: B2
- Application number: 18/034,566
- Filed: 2021-07-29
- Published: 2023-10-05
- Granted: 2024-06-04
- Earliest claimed: 2020-11-19
- Assignees: Inspur Suzhou Intelligent Technology Co., Ltd.
- Applicants: Inspur Suzhou Intelligent Technology Co., Ltd.
- Inventors: Li Wang; Fang Cao; Zhiyong Qiu; Zhenhua Guo
- Web page: https://patentgrep.com/patent/US12001960B2

## CPC

- G06N 3/063: using electronic means
- G06N 3/098: Distributed learning, e.g. federated learning
- G06F 9/5027: {the resource being a machine, e.g. CPUs, Servers, Terminals}
- G06N 3/08: Learning methods
- G06F 9/546: {Message passing systems or structures, e.g. queues}
- G06F 13/00: Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units (interface circuits for specific input/output devices G06F3/00 {; multiprogram control therefor G06F9/46}; multiprocessor systems G06F15/16 )
- G06F 2209/509: Offload
- Y02D 10/00: Energy efficient computing, e.g. low power processors, power management or thermal management
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- Y02D: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE

## Abstract

An nGraph-based graphics processing unit (GPU) backend distributed training method and system, a computer-readable storage medium, and an electronic device. The method includes: receiving a training request, and obtaining corresponding training data; obtaining a Nvidia® Collective multi-GPU Communication Library (NCCL) file by means of a system path of the NCCL file linked to an nGraph framework; invoking an NCCL communication interface configuration according to the training request to obtain a training model, the NCCL communication interface is an NCCL file-based communication operation interface located at a GPU backend of the nGraph framework; and performing GPU backend training on the training data using the training model. The present application can satisfy an urgent need of a user for performing neural network distributed training on the basis of an nGraph GPU backend, thus further improving the performance of deep learning network training.

## Claims

### Claim 1

An nGraph-based graphics processing unit (GPU) backend distributed training method, comprising:
receiving a training request, and obtaining corresponding training data;
obtaining a Collective multi-GPU Communication Library (CCL) file by means of a system path of the CCL file linked to an nGraph framework;
obtaining a communication operation function, and configuring corresponding communication operation interfaces for operations, to establish a mapping between communication operation of the nGraph-based GPU backend and corresponding communication operation in the CCL;
performing parameter parsing on the communication operation function;
establishing a function call relationship between parameters obtained by the parsing and a corresponding operation of the CCL operation, to obtain a CCL communication interface;
invoking a CCL communication interface configuration according to the training request to obtain a training model, wherein the CCL communication interface is a CCL file-based communication operation interface located at a GPU backend of the nGraph framework;
determining a distributed training type of the training model according to the training request, wherein the distributed training type comprises a multi-machine distributed training or a single-machine distributed training;
in response to the distributed training type of the training model comprising the multi-machine distributed training, performing environment training initialization by: performing message passing interface (MPI) initialization and CCL initialization; and
in response to the distributed training type of the training model comprising the single-machine distributed training, performing the environment training initialization by: performing CCL initialization; and
performing GPU backend training on the training data using the training model.

### Claim 2 (depends on 1)

The nGraph-based GPU backend distributed training method according to 52 claim 1, wherein before receiving the training request and obtaining the corresponding training data, the method further comprises:
adding the system path of the CCL file to a source code of the nGraph framework; and
modifying a compiling file of the nGraph framework, enabling a CCL function in a distributed function of the nGraph framework, and performing, in response to the CCL function being enabled, the obtaining the CCL file by means of the system path of the CCL file linked to the nGraph framework.

### Claim 3 (depends on 1)

The nGraph-based GPU backend distributed training method according to claim 1, wherein after performing GPU backend training on the training data using the training model, the method further comprises:
releasing occupied memory resources and process resources, and ending the invoking the]CCL communication interface.

### Claim 4 (depends on 1)

The nGraph-based GPU backend distributed training method according to claim 1, wherein the CCL communication interface comprises a CCL-based aggregation operation, a CCL-based broadcast operation, a CCL-based send operation and a CCL-based receive operation.

### Claim 5

A non-transitory computer-readable storage medium, storing a computer program, wherein the computer program, upon execution by a processor, is configured to cause the processor to:
receive a training request, and obtain corresponding training data;
obtain a Collective multi-GPU Communication Library (CCL) file by means of a system path of the CCL file linked to an nGraph framework;
obtain a communication operation function, and configure corresponding communication operation interfaces for operations, to establish a mapping between communication operation of the nGraph-based GPU backend and corresponding communication operation in the CCL;
perform parameter parsing on the communication operation function;
establish a function call relationship between parameters obtained by the parsing and a corresponding operation of the CCL operation, to obtain a CCL communication interface;
invoke a CCL communication interface configuration according to the training request to obtain a training model, wherein the CCL communication interface is a CCL file-based communication operation interface located at a GPU backend of the nGraph framework;
determine a distributed training type of the training model according to the training request, wherein the distributed training type comprises a multi-machine distributed training or a single-machine distributed training;
in response to the distributed training type of the training model comprising the multi-machine distributed training, perform message passing interface (MPI) initialization and CCL initialization; and
in response to the distributed training type of the training model comprising the single-machine distributed training, perform CCL initialization; and
perform GPU backend training on the training data using the training model.

### Claim 6 (depends on 5)

The non-transitory computer-readable storage medium according to claim 5, wherein the computer program, upon execution by the processor, is further configured to cause the processor to:
add the system path of the CCL file to a source code of the nGraph framework; and
modify a compiling file of the nGraph framework, enable a CCL function in a distributed function of the nGraph framework, and perform, in response to the CCL function being enabled, the obtaining the CCL file by means of the system path of the CCL file linked to the nGraph framework.

### Claim 7 (depends on 5)

The non-transitory computer-readable storage medium according to claim 5, wherein the computer program, upon execution by the processor, is further configured to cause the processor to:
release occupied memory resources and process resources, and end the invoking the CCL communication interface.

### Claim 8 (depends on 5)

The non-transitory computer-readable storage medium according to claim 5, wherein the CCL communication interface comprises a CCL-based aggregation operation, a CCL-based broadcast operation, a CCL-based send operation and a CCL-based receive operation.

### Claim 9

An electronic device, comprising:
a memory and a processor, wherein the memory stores a computer program, and the processor, upon execution of the computer program, is configured to:
receive a training request, and obtain corresponding training data;
obtain a Collective multi-GPU Communication Library (CCL) file by means of a system path of the CCL file linked to an nGraph framework;
obtain a communication operation function, and configure corresponding communication operation interfaces for operations, to establish a mapping between communication operation of the nGraph-based GPU backend and corresponding communication operation in the CCL;
perform parameter parsing on the communication operation function;
establish a function call relationship between parameters obtained by the parsing and a corresponding operation of the CCL operation, to obtain a CCL communication interface;
invoke a CCL communication interface configuration according to the training request to obtain a training model, wherein the CCL communication interface is a CCL file-based communication operation interface located at a GPU backend of the nGraph framework;
determine a distributed training type of the training model according to the training request, wherein the distributed training type comprises a multi-machine distributed training or a single-machine distributed training;
in response to the distributed training type of the training model comprising the multi-machine distributed training, perform message passing interface (MPI) initialization and CCL initialization; and
in response to the distributed training type of the training model comprising the single-machine distributed training, perform CCL initialization; and
perform GPU backend training on the training data using the training model.

### Claim 10 (depends on 9)

The electronic device according to claim 9, wherein the processor, upon execution of the computer program, is further configured to:
add the system path of the CCL file to a source code of the nGraph framework; and
modify a compiling file of the nGraph framework, enable a CCL function in a distributed function of the nGraph framework, and perform, in response to the CCL function being enabled, the obtaining the CCL file by means of the system path of the CCL file linked to the nGraph framework.

### Claim 11 (depends on 9)

The electronic device according to claim 9, wherein the processor, upon execution of the computer program, is further configured to:
release occupied memory resources and process resources, and end the invoking the CCL communication interface.