Boosting performance of ai/ml hpc workloads
AssigneeHewlett Packard Enterprise Development LP
InventorSoumitra Chatterjee
Life of this patent
Estimating the term…
Abstract
One aspect of the instant disclosure provides a system and method for managing high-performance computing (HPC) workloads. During operation, a workload management node in an HPC cluster may receive a request for executing a job for performing a machine learning task, distribute the job to a plurality of compute nodes based on the request, and configure a respective compute node of the plurality of compute nodes to store machine code associated with a function at a shared storage location. The function is part of the job, and the shared storage location is accessible to the plurality of compute nodes to allow a different compute node to execute the function without compiling source code of the function.
Claims20
A computer-implemented method for managing high-performance computing (HPC) workloads, the method comprising:
- receiving, at a computer, a request for executing a job for performing a machine learning task, the job comprising one or more functions;
- distributing, based on the request, the job to a plurality of compute nodes;
- and
- configuring a respective compute node of the plurality of compute nodes to store machine code associated with a function at a shared storage location accessible to the plurality of compute nodes to allow a different compute node to execute the function without compiling source code of the function.
A high-performance computing (HPC) system, comprising:
- a plurality of compute nodes;
- and
- an HPC workload management node comprising a processing resource and a non-transitory machine-readable storage medium comprising instructions executable by the processing resource to:
- receive a request for executing a job for performing a machine learning task, the job comprising one or more functions;
- distribute, based on the request, the job to a plurality of compute nodes;
- and
- configure a respective compute node of the plurality of compute nodes to store machine code associated with a function at a shared storage location accessible to the plurality of compute nodes to allow a different compute node to execute the function without compiling source code of the function.
A non-transitory computer-readable storage medium storing instructions to:
- receive a request for executing a high-performance computing (HPC) job for performing a machine learning task, the job comprising one or more functions;
- distribute, based on the request, the HPC job to a plurality of compute nodes;
- and
- configure a respective compute node of the plurality of compute nodes to store machine code associated with a function at a shared storage location accessible to the plurality of compute nodes to allow a different compute node to execute the function without compiling source code of the function.
Dependencies are inferred from the claim text.