Back to results
US 12,596,594 B2Granted
Reinforcement learning policy serving and training framework in production cloud systems
AssigneeInternational Business Machines Corporation
InventorsHaoran Qiu, Chen Wang, Alaa S. Youssef, Hubertus Franke, Ravishankar K. Iyer, Zbigniew Tomasz Kalbarczyk
Life of this patent
Estimating the term…
Abstract
Method and systems for online training management of reinforcement learning policy serving for cloud computing systems are discloses. An example method includes controlling a cloud computing system using a first reinforcement learning (RL) model; training the first RL model to generate a second RL model in response to one or more first criteria being satisfied; and controlling the cloud computing system using the second RL model in response to one or more second criteria being satisfied.
Claims20
A method, comprising:
- controlling resources of a cloud computing system using a first reinforcement learning (RL) model, the first RL model performs tasks to automate control of the resources while refraining from performing online training of the first RL model, wherein the first RL model, for each of the tasks, receives state information from the cloud computing system, returns an action to the cloud computing system, receives reward information from the cloud computing system, and records a first trajectory comprising the state, action, and reward for each of the tasks;
- monitoring at least one first trajectory generated while using the first RL model to identify a first trigger to start online training of the first RL model;
- performing online training of the first RL model to generate a second RL model in response to the first trigger;
- monitoring one or more second trajectories generated while performing the online training of the first RL model, to identify a second trigger to stop the online training of the first RL model;
- and
- controlling the resources of the cloud computing system using the second RL model in response to the second trigger while refraining from performing online training of the second RL model.
3 more elements
A system, comprising:
- a memory;
- and
- one or more processors coupled to the memory, the one or more processors being configured to:
- control resources of a cloud computing system using a first reinforcement learning (RL) model, the first RL model performs tasks to automate control of the resources while refraining from performing online training of the first RL model, wherein the first RL model, for each of the tasks, receives state information from the cloud computing system, returns an action to the cloud computing system, receives reward information from the cloud computing system, and records a first trajectory comprising the state, action, and reward for each of the tasks;
- monitor at least one first trajectory generated while using the first RL model, to identify a first trigger to start online training of the first RL model;
- perform online training of the first RL model to generate a second RL model in response to the first trigger;
- monitor one or more second trajectories generated while performing the online training of the first RL model, to identify a second trigger to stop the online training of the first RL model;
- and
- control the resources of the cloud computing system using the second RL model in response to the second trigger, while refraining from performing online training of the second RL model.
6 more elements
A computer program product for online training management, the computer program product comprising:
- a computer-readable storage medium having computer-readable program code embodied therewith, the computer-readable program code executable by one or more computer processors to:
- control resources of a cloud computing system using a first reinforcement learning (RL) model, the first RL model performs tasks to automate control of the resources while refraining from performing online training of the first RL model, wherein the first RL model, for each of the tasks, receives state information from the cloud computing system, returns an action to the cloud computing system, receives reward information from the cloud computing system, and records a first trajectory comprising the state, action, and reward for each of the tasks;
- monitor at least one first trajectory generated while using the first RL model, to identify a first trigger to start online training of the first RL model;
- perform online training of the first RL model to generate a second RL model in response to the first trigger;
- monitor one or more second trajectories generated while performing the online training of the first RL model, to identify a second trigger to stop the online training of the first RL model;
- and
- control the resources of the cloud computing system using the second RL model in response to the second trigger, while refraining from performing online training of the second RL model.
4 more elements
Dependencies are inferred from the claim text.