US 10,949,263 B2 · Computationally efficient reinforcement-learning-based application manager

Abstract

The current document is directed to automated reinforcement-learning-based application managers that obtain increased computational efficiency by reusing learned models and by using human-management experience to truncate state and observation vectors. Learned models of managed environments that receive component-associated inputs can be partially or completely reused for similar environments. Human managers and administrators generally use only a subset of the available metrics in managing an application, and that subset can be used as an initial subset of metrics for learning an optimal or near-optimal control policy by an automated reinforcement-learning-based application manager.

Claims20

Dependent claims

An automated reinforcement-learning-based application manager that manages a computing environment that includes one or more applications and one or more of a distributed computing system having multiple computer systems interconnected by one or more networks, a standalone computer system, and a processor-controlled user device, the reinforcement-learning based application manager comprising:

one or more processors, one or more memories, and one or more communications subsystems;
a set of actions that can be issued to the computing environment;
a set of metrics;
an iterative control process that repeatedly
selects and issues a next action, according to a control policy that uses a dimensionally reduced computational-environment vector that includes metric values and that represents a current state of the computational environment obtained by projecting a computational-environment vector into a vector subspace defined by a subset of metrics used by human application managers to control distributed applications, and
receives a reward and one of an observation and a metric vector from the computing environment, in response to execution of the issued next action, which the control process projects into the vector subspace and uses to attempt to learn an optimal or near-optimal control policy, over time.

3 more elements

A method that improves the computational efficiency of an automated reinforcement-learning-based application manager having one or more processors, one or more memories, one or more communications subsystems, a set of actions that can be issued by an iterative control process to a computing environment, controlled by the automated reinforcement-learning-based application manager, having a set of metrics, the method comprising:

enhancing the automated reinforcement-learning-based application manager by one or more of
dimensionally reducing the control process and a learning process by which the automated reinforcement-learning-based application manager learns an optimal or near-optimal control policy by projecting computational-environment vectors that represent a current state of the computational environment into a vector subspace, and
reusing a model learned during operation of a different automated reinforcement-learning-based application manager that controls a different computing environment in one or more of the control process and the learning process.

Dependencies are inferred from the claim text.

Computationally efficient reinforcement-learning-based application manager

Life of this patent

Computationally efficient reinforcement-learning-based application manager

Life of this patent

Abstract

Claims20