US 8,429,096 B1Granted

Resource isolation through reinforcement learning

AssigneeAmazon Technologies, Inc.

InventorsGokul Soundararajan, Swaminathan Sivasubramanian, Grant A. M. McAlister, Rajesh S. Sheth

Life of this patent

Estimating the term…

Loading patent…

Abstract

Systems and methods for providing resource isolation in a shared computing environment using reinforcement learning (RL) techniques are disclosed. A resource isolation mechanism may be applied in a shared storage system, or database service, that limits the resource utilization of each namespace to its specified allocation. For example, the resource isolation mechanism may be used to limit the I/O utilization of database applications in a shared computing system (e.g., a system supporting a database service) to a specified limit. In such embodiments, RL techniques may be applied to the system to automatically control the rate of queries made by an application. RL techniques, such as those based on the State-Action-Reward-State-Action (SARSA) method may be effective in controlling the I/O utilization of database applications for different workloads. RL techniques may be applied globally by the service, or may be applied to particular subscribers, applications, shared resources, namespaces, or query types.

Claims40

Dependent claims

A method, comprising:

performing, by a computer:
storing data on behalf of a plurality of subscribers in a shared storage system;
receiving input specifying one or more service level parameter values, each parameter value specifying a quantity of a respective resource that can be allocated for use in executing one or more queries in the shared storage system;
receiving a plurality of queries directed to the data;
and
executing the plurality of queries, wherein said executing comprises:
monitoring resource utilization, wherein the monitored resource utilization is resource utilization by the plurality of queries of resources for which service level parameter values have been specified, wherein said monitoring is performed during execution of the plurality of queries;
determining whether the resource utilization by the plurality of queries is consistent with the one or more specified service level parameter values;
and
in response to determining that the resource utilization by one or more of the plurality of queries is not consistent with the one or more specified service level parameter values, attempting to correct the determined inconsistency, wherein said attempting comprises applying a reinforcement learning technique to automatically update an execution parameter of at least one of the plurality of queries, wherein an execution parameter is a modifiable parameter affecting execution of the plurality of queries, and wherein the update is performed during execution of the plurality of queries.

7 more elements

A computing system, comprising:

one or more computing nodes, wherein each of the one or more computing nodes comprises at least one processor and a memory, and wherein the one or more computing nodes are configured to collectively implement a database service, wherein the database service is configured to:
store data on behalf of a plurality of subscribers to the database service;
receive input specifying one or more service level parameter values, each parameter value specifying a quantity of a respective resource that can be allocated for use in executing one or more queries by the database service;
receive a plurality of queries directed to the data;
and
execute the plurality of queries, wherein to execute the plurality of queries, the database service is further configured to:
monitor resource utilization, wherein the monitored resource utilization is resource utilization by the plurality of queries of resources for which service level parameter values have been specified, wherein said monitoring is performed during execution of the plurality of queries;
determine whether the resource utilization by the plurality of queries is consistent with the one or more specified service level parameter values;
and
in response to determining that the resource utilization by one or more of the plurality of queries is not consistent with the one or more specified service level parameter values, attempt to correct the determined inconsistency, wherein to attempt to correct the determined inconsistency, the database service is configured to apply a reinforcement learning technique to automatically update an execution parameter of at least one of the plurality of queries, wherein an execution parameter is a modifiable parameter affecting execution of the plurality of queries, and wherein the update is performed during execution of the plurality of queries.

7 more elements

A computing system, comprising:

one or more processors;
and
a memory coupled to the one or more processors and configured to implement a shared storage system;
wherein the memory is further configured to store program instructions executable by the one or more processors to implement:
storing data in the shared storage system on behalf of a plurality of subscribers;
receiving input specifying one or more service level parameter values, each parameter value specifying a quantity of a respective resource that can be allocated for use in executing one or more queries in the shared storage system;
receiving a plurality of queries directed to the data;
and
executing the plurality of queries, wherein to execute the plurality of queries, the program instructions are further executable to implement:
monitoring resource utilization, wherein the monitored resource utilization is resource utilization by the plurality of queries of resources for which service level parameter values have been specified, wherein said monitoring is performed during execution of the plurality of queries;
determining whether the resource utilization by the plurality of queries is consistent with the one or more specified service level parameter values;
and
in response to determining that the resource utilization by one or more of the plurality of queries is not consistent with the one or more specified service level parameter values, attempting to correct the determined inconsistency, wherein said attempting comprises applying a reinforcement learning technique to automatically update an execution parameter of at least one of the plurality of queries, wherein an execution parameter is a modifiable parameter affecting execution of the plurality of queries, and wherein the update is performed during execution of the plurality of queries.

10 more elements

A non-transitory, computer-readable storage medium storing program instructions computer-executable to implement:

storing data in a shared storage system on behalf of a plurality of subscribers;
receiving input specifying one or more service level parameter values, each parameter value specifying a quantity of a respective resource that can be allocated for use in executing one or more queries in the shared storage system;
receiving a plurality of queries directed to the data;
and
executing the plurality of queries, wherein to execute the plurality of queries, the program instructions are further computer-executable to implement:
monitoring resource utilization, wherein the monitored resource utilization is resource utilization by the plurality of queries of resources for which service level parameter values have been specified, wherein said monitoring is performed during execution of the plurality of queries;
determining whether the resource utilization by the plurality of queries is consistent with the one or more specified service level parameter values;
and
in response to determining that the resource utilization by one or more of the plurality of queries is not consistent with the one or more specified service level parameter values, attempting to correct the determined inconsistency, wherein said attempting comprises applying a reinforcement learning technique to automatically update an execution parameter of at least one of the plurality of queries, wherein an execution parameter is a modifiable parameter affecting execution of the plurality of queries, and wherein the update is performed during execution of the plurality of queries.

6 more elements

A computer-implemented method for providing computing services on behalf of a plurality of subscribers in a shared computing system, comprising:

receiving input specifying one or more service level parameter values, each parameter value specifying a quantity of a respective resource of the shared computing system that can be allocated to a given one of the plurality of subscribers for use in executing one or more computing requests;
receiving a plurality of computing requests;
and
executing the plurality of computing requests, wherein said executing comprises:
monitoring resource utilization, wherein the monitored resource utilization is resource utilization by the plurality of computing requests of resources for which service level parameter values have been specified, wherein said monitoring is performed during execution of the plurality of computing requests;
determining whether the resource utilization by the plurality of computing requests is consistent with the one or more specified service level parameter values;
in response to determining that the resource utilization by one or more of the plurality of queries is not consistent with the one or more specified service level parameter values, attempting to correct the determined inconsistency, wherein said attempting comprises:
applying a reinforcement learning technique to automatically update an execution parameter of at least one of the plurality of computing requests, wherein an execution parameter is a modifiable parameter affecting execution of the plurality of computing requests, and wherein the update is performed during execution of the plurality of computing requests;
determining a reward value of said update;
and
modifying a previously estimated reward value of said update;
wherein the modified reward value is usable in selecting one of a plurality of available actions in a subsequent application of the reinforcement learning technique;
and
wherein each of the plurality of actions comprises a specified update to a given execution parameter.

11 more elements

Dependencies are inferred from the claim text.