Automated reinforcement-learning-based application manager that uses local agents
AssigneeVMware, Inc.
InventorsDev Nag, Yanislav Yankov, Dongni Wang, Gregory T. Burk, Nicholas Mark Grant Stephen
Life of this patent
Estimating the term…
Abstract
The current document is directed to automated reinforcement-learning-based application managers that use local agents. Local agents provide finer-granularity monitoring of an application or application subcomponents and provide continued application management in the event of interruption of network traffic between an automated reinforcement-learning-based application manager and the application or application subcomponents managed by the automated reinforcement-learning-based application manager.
Claims21
An automated reinforcement-learning-based application manager that manages a computing environment that includes one or more applications, one or more local agents, and one or more of a distributed computing system having multiple computer systems interconnected by one or more networks, a standalone computer system, and a processor-controlled user device, the reinforcement-learning based application manager comprising:
- one or more processors, one or more memories, and one or more communications subsystems;
- a set of actions that can be issued to the computing environment;
- and
- an iterative control process that repeatedly
- selects and issues a next action to one or more of the one or more local agents according to a control policy that uses a state vector that represents a current state of the computing system,
- receives, from one or more of the one or more local agents in response to execution of the issued next action, one of an observation and state vector, from which the control process generates a current state and a reward, which the control process uses to attempt to learn an optimal or near-optimal control policy.
A method that provides fault tolerance and fine-granularity control to an automated reinforcement-learning-based application manager that manages a computing environment that includes one or more applications and one or more of a distributed computing system having multiple computer systems interconnected by one or more networks, a standalone computer system, and a processor-controlled user device, the automated reinforcement-learning-based application manager having one or more processors, one or more memories, one or more communications subsystems, and a set of actions that can be issued to the computing environment, the method comprising:
- iteratively
- selecting and issuing a next action to one or more of the one or more local agents according to a control policy that uses a state vector that represents a current state of the computing system,
- receiving, from one or more of the one or more local agents in response to execution of the issued next action, one of an observation and state vector, from which the control process generates a current state and a reward, which the control process uses to attempt to learn an optimal or near-optimal control policy.
A physical data-storage device encoded with computer instructions that, when executed by one or more processors of a computer system that implements an automated reinforcement-learning-based application manager having one or more processors, one or more memories, one or more communications subsystems, a set of actions that can be issued to a computing environment, controls the automated reinforcement-learning-based application manager to provide fault-tolerant operation and fine-grain control by:
- iteratively
- selecting and issuing a next action to one or more of the one or more local agents according to a control policy that uses a state vector that represents a current state of the computing system,
- receiving, from one or more of the one or more local agents in response to execution of the issued next action, one of an observation and state vector, from which the control process generates a current state and a reward, which the control process uses to attempt to learn an optimal or near-optimal control policy.
Dependencies are inferred from the claim text.