# US 2019/0312800 A1 · Method, apparatus and system for real-time optimization of computer-implemented application operations using machine learning techniques

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0312800 A1
- Document ID: US20190312800A1
- Status: Published application
- Kind: A1
- Application number: 16/435,516
- Filed: 2019-06-09
- Published: 2019-10-10
- Earliest claimed: 2015-07-27
- Assignees: Datagrid Systems, Inc.
- Applicants: Datagrid Systems, Inc.
- Inventors: Ross Schibler; Peter Nickolov; Stephen Quintero
- Web page: https://patentgrep.com/patent/US20190312800A1

## CPC

- G06F 8/65: Updates (security arrangements therefor G06F21/57)
- G06F 9/44505: {Configuring for program initiating, e.g. using registry, configuration files}
- G06F 9/45558: {Hypervisor-specific management and integration aspects}
- H04L 41/082: the condition being updates or upgrades of network functionality
- H04L 41/0823: characterised by the purposes of a change of settings, e.g. optimising configuration for enhancing reliability (for optimising operational conditions of wireless networks H04W24/02)
- H04L 41/0869: Validating the configuration within one network element
- H04L 41/16: using machine learning or artificial intelligence
- H04L 43/0817: by checking functioning
- H04L 63/1433: {Vulnerability analysis}
- G06F 8/658: Incremental updates; Differential updates
- G06F 2009/45591: {Monitoring or debugging support}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

Various aspects described or referenced herein are directed to different methods, systems, and computer program products for facilitating real-time optimization of computer-implemented application operations using machine learning techniques.

## Claims

### Claim 1

A computer implemented method for optimizing a mutable runtime configuration of a first application hosted at a remote networked environment, the remote networked environment being communicatively coupled to a computer network, the computer network including an Optimizer System, the Optimizer System including a first set of optimizer algorithms, the method comprising causing at least one processor to execute instructions stored in non-transient memory to:
cause at least one network device to initiate a first measurement of a first operational metric of the first application while the first application is operating in accordance with a first runtime configuration;
cause the at least one network device to transmit first measurement information to the Optimizer System, where the first measurement information relates to the first measurement of the first operational metric of the first application;
calculate, using the first measurement information, a first score in relation to a first optimization objective, the first score being calculated using a first scoring function;
determine, at the Optimizer System, a first set of updated application settings relating to the mutable runtime configuration of the first application;
cause, using the at least one network device, the first set of updated application settings to be deployed at the first application to thereby cause the first application to operate in accordance with a second runtime configuration;
cause the at least one network device to initiate a second measurement of the first operational metric of the first application while the first application is operating in accordance with the second runtime configuration;
cause the at least one network device to transmit second measurement information to the Optimizer System, where the second measurement information relates to the second measurement of the first operational metric of the first application;
calculate, using the second measurement information, a second score in relation to the first optimization objective, the second score being calculated using the first scoring function;
compute, using at least the second score, a first reward;
update the first set of optimization algorithms using information relating to the first reward;
select, from the first set of optimization algorithms, a first optimization algorithm to be used for determining a second set of updated application settings relating to the mutable runtime configuration of the first application;
determine, using the first optimization algorithm, a second set of updated application settings relating to the mutable runtime configuration of the first application;
cause, using the at least one network device, the second set of updated application settings to be deployed at the first application to thereby cause the first application to operate in accordance with a third runtime configuration;
cause the at least one network device to initiate a third measurement of the first operational metric of the first application while the first application is operating in accordance with the third runtime configuration;
cause the at least one network device to transmit third measurement information to the Optimizer System, where the third measurement information relates to the third measurement of the first operational metric of the first application;
calculate, using the third measurement information, a third score in relation to the first optimization objective, the third score being calculated using the first scoring function;
compute, using at least the third score, a second reward;
update the first set of optimization algorithms using information relating to the second reward;
select, from the first set of optimization algorithms, a second optimization algorithm to be used for determining a third set of updated application settings relating to the mutable runtime configuration of the first application;
determine, using the second optimization algorithm, a third set of updated application settings relating to the mutable runtime configuration of the first application;
cause, using the at least one network device, the third set of updated application settings to be deployed at the first application to thereby cause the first application to operate in accordance with a fourth runtime configuration; and
determine, at the Optimizer System, if additional cycles of optimization adjustment are to be performed for the first application.

### Claim 2 (depends on 1)

The computer implemented method of claim 1 further comprising causing, if it is determined that additional cycles of optimization adjustment are to be performed for the first application, the at least one processor to execute additional instructions to:
cause the at least one network device to initiate a fourth measurement of the first operational metric of the first application while the first application is operating in accordance with the fourth runtime configuration;
cause the at least one network device to transmit forth measurement information to the Optimizer System, where the fourth measurement information relates to the fourth measurement of the first operational metric of the first application;
calculate, using the fourth measurement information, a fourth score in relation to the first optimization objective, the fourth score being calculated using the first scoring function;
compute, using at least the fourth score, a third reward;
update the first set of optimization algorithms using information relating to the third reward;
select, from the first set of optimization algorithms, a third optimization algorithm to be used for determining a fourth set of updated application settings relating to the mutable runtime configuration of the first application;
determine, using the third optimization algorithm, a fourth set of updated application settings relating to the mutable runtime configuration of the first application; and
cause, using the at least one network device, the fourth set of updated application settings to be deployed at the first application to thereby cause the first application to operate in accordance with a fifth runtime configuration.

### Claim 3 (depends on 1)

The computer implemented method of claim 1 wherein the at least one network component includes a servo component deployed at the remote networked environment and configured or designed to implement instructions received from the Optimizer System, and to initiate interactions with the first application in response to the received instructions.

### Claim 4 (depends on 1)

The computer implemented method of claim 1 wherein the at least one network component includes a servo component deployed at the Optimizer System and configured or designed to implement instructions generated by the Optimizer System and to initiate interactions with the first application in response to the instructions.

### Claim 5 (depends on 1)

The computer implemented method of claim 1 further comprising causing the at least one processor to execute additional instructions to:
calculate, using the first measurement information, a first performance indicator of the first application, the first performance indicator being representative of a first performance of the first application while operating in accordance with the first runtime configuration;
calculate, using information relating to the first runtime configuration, a first cost indicator of the first application, the first cost indicator being representative of a first cost of resources utilized for operating the first application in accordance with the first runtime configuration;
wherein the first score is calculated using the first performance indicator and first cost indicator;
calculate, using the second measurement information, a second performance indicator of the first application, the second performance indicator being representative of a second performance of the first application while operating in accordance with the second runtime configuration;
calculate, using information relating to the second runtime configuration, a second cost indicator of the first application, the second cost indicator being representative of a second cost of resources utilized for operating the first application in accordance with the second runtime configuration; and
wherein the second score is calculated using the second performance indicator and second cost indicator.

### Claim 6 (depends on 1)

The computer implemented method of claim 1 wherein the first reward corresponds to the second score.

### Claim 7 (depends on 1)

The computer implemented method of claim 1 wherein the first reward is calculated based on a comparison of the second score and the first score.

### Claim 8 (depends on 1)

The computer implemented method of claim 1 further comprising causing the at least one processor to execute additional instructions to:
calculate, using the first measurement information, a first performance measurement of the first application;
calculate, using information relating to the first runtime configuration, a first cost of the application;
wherein the first score is calculated using the first performance measurement and first cost; and
wherein the first scoring function corresponds to a scoring function selected from a group consisting of:
performance measurement/cost;
performance measurement W1/cost, where W1 represents a weighted value;
performance measurement, where cost is represented as constant;
performance measurement bounded by a maximum cost; and
cost while maintaining a minimum performance measurement value.

### Claim 9 (depends on 1)

The computer implemented method of claim 1:
wherein at least one set of updated application settings is selected from a group consisting of:
at least one virtual machine associated with the first application;
at least one container associated with the first application;
at least one CPU core associated with the first application;
at least one memory associated with the first application;
network bandwidth associated with the first application;
at least one provisioned disk TOPS associated with the first application;
at least one resource setting associated with the first application; and
number of replicas of a component deployed at the first application.

### Claim 10 (depends on 1)

The computer implemented method of claim 1:
wherein at least one set of updated application settings is selected from a group consisting of:
the number of Apache worker threads associated with the first application;
My SQL memory pool size associated with the first application;
kernel tuning parameters associated with the first application;
number of virtualized components of the first application which are co-located on a same physical machine; and
number of virtualized container components of the first application which are co-located on a same host.

### Claim 11 (depends on 1)

The computer implemented method of claim 1 wherein at least one selected optimization algorithm corresponds to a reinforced learning algorithm configured or designed to employ Q-learning using a neural network as a Q function.

### Claim 12 (depends on 1)

The computer implemented method of claim 1:
wherein the first optimization algorithm corresponds to a first type of optimization algorithm selected from a group consisting of: a reinforced learning algorithm configured or designed to employ Q-learning using a neural network as a Q function, a Bayesian algorithm, an Evolutionary algorithm, an Ouch heuristic algorithm, a Stochastic algorithm, and a Bruteforce algorithm;
wherein the second optimization algorithm corresponds to a second type of optimization algorithm selected from a group consisting of: a reinforced learning algorithm configured or designed to employ Q-learning using a neural network as a Q function, a Bayesian algorithm, an Evolutionary algorithm, an Ouch heuristic algorithm, a Stochastic algorithm, and a Bruteforce algorithm; and
wherein the first type of optimization algorithm is different from the second type of optimization algorithm.

### Claim 13 (depends on 1)

The computer implemented method of claim 1 further comprising causing the at least one processor to execute instructions to cause at least one set of updated application settings to be deployed at the first application while the first application is running in a live production environment.

### Claim 14 (depends on 1)

The computer implemented method of claim 1 further comprising causing the at least one processor to execute instructions to cause at least one set of updated application settings to be deployed at the first application while the first application is running in a test bed environment.

### Claim 15 (depends on 1)

The computer implemented method of claim 1 further comprising causing the at least one processor to execute instructions to cause at least one set of updated application settings to be deployed at the first application while the first application is running in a canary environment.

### Claim 16 (depends on 1)

The computer implemented method of claim 1 wherein the method is implemented as a SaaS service which is configured or designed to utilize the Optimizer System to remotely and securely optimize the first application.

### Claim 17

A computer implemented system for optimizing a mutable runtime configuration of a first application hosted at a remote networked environment, the remote networked environment being communicatively coupled to a computer network, the computer network including an Optimizer System, the Optimizer System including a first set of optimizer algorithms, the system comprising:
at least one non-transient memory;
at least one processor operable to execute a plurality of instructions stored in the at least one non-transient memory to:
cause at least one network device to initiate a first measurement of a first operational metric of the first application while the first application is operating in accordance with a first runtime configuration;
cause the at least one network device to transmit first measurement information to the Optimizer System, where the first measurement information relates to the first measurement of the first operational metric of the first application;
calculate, using the first measurement information, a first score in relation to a first optimization objective, the first score being calculated using a first scoring function;
determine, at the Optimizer System, a first set of updated application settings relating to the mutable runtime configuration of the first application;
cause, using the at least one network device, the first set of updated application settings to be deployed at the first application to thereby cause the first application to operate in accordance with a second runtime configuration;
cause the at least one network device to initiate a second measurement of the first operational metric of the first application while the first application is operating in accordance with the second runtime configuration;
cause the at least one network device to transmit second measurement information to the Optimizer System, where the second measurement information relates to the second measurement of the first operational metric of the first application;
calculate, using the second measurement information, a second score in relation to the first optimization objective, the second score being calculated using the first scoring function;
compute, using the second and first scores, a first reward;
update the first set of optimization algorithms using information relating to the first reward;
select, from the first set of optimization algorithms, a first optimization algorithm to be used for determining a second set of updated application settings relating to the mutable runtime configuration of the first application;
determine, using the first optimization algorithm, a second set of updated application settings relating to the mutable runtime configuration of the first application;
cause, using the at least one network device, the second set of updated application settings to be deployed at the first application to thereby cause the first application to operate in accordance with a third runtime configuration;
cause the at least one network device to initiate a third measurement of the first operational metric of the first application while the first application is operating in accordance with the third runtime configuration;
cause the at least one network device to transmit third measurement information to the Optimizer System, where the third measurement information relates to the third measurement of the first operational metric of the first application;
calculate, using the third measurement information, a third score in relation to the first optimization objective, the third score being calculated using the first scoring function;
compute, using the second and third scores, a second reward;
update the first set of optimization algorithms using information relating to the second reward;
select, from the first set of optimization algorithms, a second optimization algorithm to be used for determining a third set of updated application settings relating to the mutable runtime configuration of the first application;
determine, using the second optimization algorithm, a third set of updated application settings relating to the mutable runtime configuration of the first application;
cause, using the at least one network device, the third set of updated application settings to be deployed at the first application to thereby cause the first application to operate in accordance with a fourth runtime configuration; and
determine, at the Optimizer System, if additional cycles of optimization adjustment are to be performed for the first application.

### Claim 18 (depends on 17)

The computer implemented system of claim 17 being further operable to cause, if it is determined that additional cycles of optimization adjustment are to be performed for the first application, the at least one processor to execute additional instructions to:
causing the at least one processor to execute additional instructions to:
cause the at least one network device to initiate a fourth measurement of the first operational metric of the first application while the first application is operating in accordance with the fourth runtime configuration;
cause the at least one network device to transmit forth measurement information to the Optimizer System, where the fourth measurement information relates to the fourth measurement of the first operational metric of the first application;
calculate, using the fourth measurement information, a fourth score in relation to the first optimization objective, the fourth score being calculated using the first scoring function;
compute, using the third and fourth scores, a third reward;
update the first set of optimization algorithms using information relating to the third reward;
select, from the first set of optimization algorithms, a third optimization algorithm to be used for determining a fourth set of updated application settings relating to the mutable runtime configuration of the first application;
determine, using the third optimization algorithm, a fourth set of updated application settings relating to the mutable runtime configuration of the first application; and
cause, using the at least one network device, the fourth set of updated application settings to be deployed at the first application to thereby cause the first application to operate in accordance with a fifth runtime configuration.

### Claim 19 (depends on 17)

The computer implemented system of claim 17 wherein the at least one network component includes a servo component deployed at the remote networked environment and configured or designed to implement instructions received from the Optimizer System, and to initiate interactions with the first application in response to the received instructions.

### Claim 20 (depends on 17)

The computer implemented system of claim 17 wherein the at least one network component includes a servo component deployed at the Optimizer System and configured or designed to implement instructions generated by the Optimizer System and to initiate interactions with the first application in response to the instructions.

### Claim 21 (depends on 17)

The computer implemented system of claim 17 being further operable to cause the at least one processor to execute additional instructions to:
calculate, using the first measurement information, a first performance indicator of the first application, the first performance indicator being representative of a first performance of the first application while operating in accordance with the first runtime configuration;
calculate, using information relating to the first runtime configuration, a first cost indicator of the first application, the first cost indicator being representative of a first cost of resources utilized for operating the first application in accordance with the first runtime configuration;
wherein the first score is calculated using the first performance indicator and first cost indicator;
calculate, using the second measurement information, a second performance indicator of the first application, the second performance indicator being representative of a second performance of the first application while operating in accordance with the second runtime configuration;
calculate, using information relating to the second runtime configuration, a second cost indicator of the first application, the second cost indicator being representative of a second cost of resources utilized for operating the first application in accordance with the second runtime configuration; and
wherein the second score is calculated using the second performance indicator and second cost indicator.

### Claim 22 (depends on 17)

The computer implemented system of claim 17 wherein the first reward corresponds to the second score.

### Claim 23 (depends on 17)

The computer implemented system of claim 17 wherein the first reward is calculated based on a comparison of the second score and the first score.

### Claim 24 (depends on 17)

The computer implemented system of claim 17 being further operable to cause the at least one processor to execute additional instructions to:
calculate, using the first measurement information, a first performance measurement of the first application;
calculate, using information relating to the first runtime configuration, a first cost of the application;
wherein the first score is calculated using the first performance measurement and first cost; and
wherein the first scoring function corresponds to a scoring function selected from a group consisting of:
performance measurement/cost;
performance measurement W1/cost, where W1 represents a weighted value;
performance measurement, where cost is represented as constant;
performance measurement bounded by a maximum cost; and
cost while maintaining a minimum performance measurement value.

### Claim 25 (depends on 17)

The computer implemented system of claim 17:
wherein at least one set of updated application settings is selected from a group consisting of:
at least one virtual machine associated with the first application;
at least one container associated with the first application;
at least one CPU core associated with the first application;
at least one memory associated with the first application;
network bandwidth associated with the first application;
at least one provisioned disk TOPS associated with the first application;
at least one resource setting associated with the first application; and
number of replicas of a component deployed at the first application.

### Claim 26 (depends on 17)

The computer implemented system of claim 17:
wherein at least one set of updated application settings is selected from a group consisting of:
the number of Apache worker threads associated with the first application;
My SQL memory pool size associated with the first application;
kernel tuning parameters associated with the first application;
number of virtualized components of the first application which are co-located on a same physical machine; and
number of virtualized container components of the first application which are co-located on a same host.

### Claim 27 (depends on 17)

The computer implemented system of claim 17 wherein at least one selected optimization algorithm corresponds to a reinforced learning algorithm configured or designed to employ Q-learning using a neural network as a Q function.

### Claim 28 (depends on 17)

The computer implemented system of claim 17:
wherein the first optimization algorithm corresponds to a first type of optimization algorithm selected from a group consisting of: a reinforced learning algorithm configured or designed to employ Q-learning using a neural network as a Q function, a Bayesian algorithm, an Evolutionary algorithm, an Ouch heuristic algorithm, a Stochastic algorithm, and a Bruteforce algorithm;
wherein the second optimization algorithm corresponds to a second type of optimization algorithm selected from a group consisting of: a reinforced learning algorithm configured or designed to employ Q-learning using a neural network as a Q function, a Bayesian algorithm, an Evolutionary algorithm, an Ouch heuristic algorithm, a Stochastic algorithm, and a Bruteforce algorithm; and
wherein the first type of optimization algorithm is different from the second type of optimization algorithm.

### Claim 29 (depends on 17)

The computer implemented system of claim 17 being further operable to cause the at least one processor to execute instructions to cause at least one set of updated application settings to be deployed at the first application while the first application is running in a live production environment.

### Claim 30 (depends on 17)

The computer implemented system of claim 17 being further operable to cause the at least one processor to execute instructions to cause at least one set of updated application settings to be deployed at the first application while the first application is running in a test bed environment.

### Claim 31 (depends on 17)

The computer implemented system of claim 17 being further operable to cause the at least one processor to execute instructions to cause at least one set of updated application settings to be deployed at the first application while the first application is running in a canary environment.

### Claim 32 (depends on 17)

The computer implemented system of claim 17 wherein the system is implemented as a SaaS service which is configured or designed to utilize the Optimizer System to remotely and securely optimize the first application.

### Claim 33

A non-transitory computer usable medium for use in a computer network, the computer network including a first application hosted at a remote networked environment, and including an Optimizer System configured to access a first set of optimizer algorithms, the computer network further including at least one processor, the computer usable medium having computer readable code embodied therein, the computer readable code comprising computer code for causing at least one processor to execute instructions stored in at least one memory to:
cause at least one network device to initiate a first measurement of a first operational metric of the first application while the first application is operating in accordance with a first runtime configuration;
cause the at least one network device to transmit first measurement information to the Optimizer System, where the first measurement information relates to the first measurement of the first operational metric of the first application;
calculate, using the first measurement information, a first score in relation to a first optimization objective, the first score being calculated using a first scoring function;
determine, at the Optimizer System, a first set of updated application settings relating to the mutable runtime configuration of the first application;
cause, using the at least one network device, the first set of updated application settings to be deployed at the first application to thereby cause the first application to operate in accordance with a second runtime configuration;
cause the at least one network device to initiate a second measurement of the first operational metric of the first application while the first application is operating in accordance with the second runtime configuration;
cause the at least one network device to transmit second measurement information to the Optimizer System, where the second measurement information relates to the second measurement of the first operational metric of the first application;
calculate, using the second measurement information, a second score in relation to the first optimization objective, the second score being calculated using the first scoring function;
compute, using the second and first scores, a first reward;
update the first set of optimization algorithms using information relating to the first reward;
select, from the first set of optimization algorithms, a first optimization algorithm to be used for determining a second set of updated application settings relating to the mutable runtime configuration of the first application;
determine, using the first optimization algorithm, a second set of updated application settings relating to the mutable runtime configuration of the first application;
cause, using the at least one network device, the second set of updated application settings to be deployed at the first application to thereby cause the first application to operate in accordance with a third runtime configuration;
cause the at least one network device to initiate a third measurement of the first operational metric of the first application while the first application is operating in accordance with the third runtime configuration;
cause the at least one network device to transmit third measurement information to the Optimizer System, where the third measurement information relates to the third measurement of the first operational metric of the first application;
calculate, using the third measurement information, a third score in relation to the first optimization objective, the third score being calculated using the first scoring function;
compute, using the second and third scores, a second reward;
update the first set of optimization algorithms using information relating to the second reward;
select, from the first set of optimization algorithms, a second optimization algorithm to be used for determining a third set of updated application settings relating to the mutable runtime configuration of the first application;
determine, using the second optimization algorithm, a third set of updated application settings relating to the mutable runtime configuration of the first application;
cause, using the at least one network device, the third set of updated application settings to be deployed at the first application to thereby cause the first application to operate in accordance with a fourth runtime configuration; and
determine, at the Optimizer System, if additional cycles of optimization adjustment are to be performed for the first application.
