Back to results
US 12,724,645 B2Granted
Virtualized computing resource management for machine learning model-based processing in computing environment
AssigneeDell Products L.P.
InventorVictor Fong
Life of this patent
Estimating the term…
Abstract
Techniques are disclosed for virtualized computing resource management for machine learning model-based processing in a computing environment. For example, a method maintains one or more virtualized computing resources, wherein each of the one or more virtualized computing resources is created and one or more initializations are caused to be performed. After creation and performance of the one or more initializations, each of the one or more virtualized computing resources is placed in an idle state. The method then receives a machine learning model-based request, and removes at least one of the one or more virtualized computing resources from the idle state to process the machine learning model-based request.
Claims20
A method, comprising:
- determining a given number of virtualized computing resources to maintain in an idle state in a system, wherein the given number of virtualized computing resources to maintain in an idle state is determined based on available computational resources and traffic volume of the system;
- maintaining the given number of the virtualized computing resources in an idle state in a standby pool of the system, wherein maintaining comprises creating the given number of virtualized computing resources, and performing initialization operations for each of the virtualized computing resources created, wherein the initialization operations comprise:
- initializing a given virtualized computing resource on an operating system of a compute node of the system;
- initializing a machine learning framework for use by the given virtualized computing resource;
- initializing a hardware accelerator processing device for use by the given virtualized computing resource, wherein initializing the hardware accelerator processing device comprises creating a placeholder session to register the given virtualized computing resource on a memory space of the hardware accelerator processing device;
- and
- initializing the given virtualized computing resource into an idle state;
- receiving a machine learning model-based request;
- and
- in response to receiving the machine learning model-based request:
- waking up and removing from the standby pool one or more of the virtualized computing resources from the idle state to process the machine learning model-based request;
- and
- creating and initializing one or more new virtualized computing resources into an idle state to replace the virtualized computing resources removed from the standby pool;
- wherein the method is performed by at least one processor accessing and executing program instructions stored in at least one memory.
11 more elements
An apparatus, comprising:
- at least one processor and at least one memory storing computer program instructions wherein, when the at least one processor executes the computer program instructions, the apparatus is configured to:
- determine a given number of virtualized computing resources to maintain in an idle state in a system, wherein the given number of virtualized computing resources to maintain in an idle state is determined based on available computational resources and traffic volume of the system;
- maintaining the given number of the virtualized computing resources in an idle state in a standby pool of the system, wherein maintaining comprises creating the given number of virtualized computing resources, and performing initialization operations for each of the virtualized computing resources created, wherein the initialization operations comprise:
- initializing a given virtualized computing resource on an operating system of a compute node of the system;
- initializing a machine learning framework for use by the given virtualized computing resource;
- initializing a hardware accelerator processing device for use by the given virtualized computing resource, wherein initializing the hardware accelerator processing device comprises creating a placeholder session to register the given virtualized computing resource on a memory space of the hardware accelerator processing device;
- and
- initializing the given virtualized computing resource into an idle state;
- receive a machine learning model-based request;
- and
- in response to receiving the machine learning model-based request:
- wake up and remove from the standby pool one or more of the virtualized computing resources from the idle state to process the machine learning model-based request;
- and
- create and initialize one or more new virtualized computing resources into an idle state to replace the virtualized computing resources removed from the standby pool.
11 more elements
A computer program product stored on a non-transitory computer-readable medium and comprising machine executable instructions, the machine executable instructions, when executed, causing a processing device to perform steps of:
- determining a given number of virtualized computing resources to maintain in an idle state in a system, wherein the given number of virtualized computing resources to maintain in an idle state is determined based on available computational resources and traffic volume of the system;
- maintaining the given number of the virtualized computing resources in an idle state in a standby pool of the system, wherein maintaining comprises creating the given number of virtualized computing resources, and performing initialization operations for each of the virtualized computing resources created, wherein the initialization operations comprise:
- initializing a given virtualized computing resource on an operating system of a compute node of the system;
- initializing a machine learning framework for use by the given virtualized computing resource;
- initializing a hardware accelerator processing device for use by the given virtualized computing resource, wherein initializing the hardware accelerator processing device comprises creating a placeholder session to register the given virtualized computing resource on a memory space of the hardware accelerator processing device;
- and
- initializing the given virtualized computing resource into an idle state;
- receiving a machine learning model-based request;
- and
- in response the receiving the machine learning model-based request:
- waking up and removing from the standby pool one or more of the virtualized computing resources from the idle state to process the machine learning model-based request;
- and
- creating and initializing one or more new virtualized computing resources into an idle state to replace the virtualized computing resources removed from the standby pool to process the machine learning model-based request.
10 more elements
Dependencies are inferred from the claim text.