US 8,429,097 B1Granted

Resource isolation using reinforcement learning and domain-specific constraints

AssigneeAmazon Technologies, Inc.

InventorsSwaminathan Sivasubramanian, Parikshit S. Pol

Life of this patent

Estimating the term…

Loading patent…

Abstract

Systems and methods for providing resource isolation in a shared computing environment using reinforcement learning (RL) techniques are disclosed. A resource isolation mechanism may be applied in a shared storage system, or database service, that limits the resource utilization of each namespace to its specified allocation. For example, the mechanism may be used to limit the I/O utilization of database applications in a shared computing system to a specified limit, and may apply RL techniques, along with domain-specific constraints, to the system to automatically control the rate of queries made by an application. RL techniques, such as those based on the State-Action-Reward-State-Action (SARSA) method, may be effective in controlling resource utilization for different workloads, and may be enhanced by their application in conjunction with domain-specific business rules. RL techniques may be applied globally by the service, or may be applied to particular subscribers, applications, shared resources, namespaces, or query types.

Claims33

Dependent claims

A method, comprising:

performing, by a computer:
storing data on behalf of a plurality of subscribers in a shared storage system;
receiving input specifying one or more service level parameter values, each parameter value specifying a quantity of a respective resource that can be allocated for use in executing one or more queries in the shared storage system;
receiving a plurality of queries directed to the data;
and
executing the plurality of queries, wherein said executing comprises:
monitoring resource utilization, wherein the monitored resource utilization is resource utilization by the plurality of queries of resources for which service level parameter values have been specified, wherein said monitoring is performed during execution of the plurality of queries;
determining whether the resource utilization by the plurality of queries is consistent with the one or more specified service level parameter values;
and
in response to determining that the resource utilization by one or more of the plurality of queries is not consistent with the one or more specified service level parameter values, attempting to correct the determined inconsistency, wherein said attempting comprises applying a reinforcement learning technique in conjunction with one or more domain-specific constraints to automatically update an execution parameter of at least one of the plurality of queries such that a result of updating the execution parameter is consistent with the one or more domain-specific constraints, wherein an execution parameter is a modifiable parameter affecting execution of the plurality of queries, and wherein the update is performed during execution of the plurality of queries.

7 more elements

A computing system, comprising one or more computing nodes, wherein each of the one or more computing nodes comprises at least one processor and a memory, and wherein the one or more computing nodes are configured to collectively implement a database service, wherein the database service is configured to:

store data on behalf of a plurality of subscribers to the database service;
receive input specifying one or more service level parameter values, each parameter value specifying a quantity of a respective resource that can be allocated for use in executing one or more queries by the database service;
receive a plurality of queries directed to the data;
and
execute the plurality of queries, wherein to execute the plurality of queries, the database service is configured to:
monitor resource utilization, wherein the monitored resource utilization is resource utilization by the plurality of queries of resources for which service level parameter values have been specified, wherein said monitoring is performed during execution of the plurality of queries;
determine whether the resource utilization by the plurality of queries is consistent with the one or more specified service level parameter values;
and
in response to determining that the resource utilization by one or more of the plurality of queries is not consistent with the one or more specified service level parameter values, attempt to correct the determined inconsistency, wherein to attempt to correct the determined inconsistency, the database service is configured to apply a reinforcement learning technique in conjunction with one or more domain-specific constraints to automatically update an execution parameter of at least one of the plurality of queries such that a result of updating the execution parameter is consistent with the one or more domain-specific constraints, wherein an execution parameter is a modifiable parameter affecting execution of the plurality of queries, and wherein the update is performed during execution of the plurality of queries.

6 more elements

A computing system, comprising:

one or more processors;
and
a memory coupled to the one or more processors and configured to implement a shared storage system;
wherein the memory is further configured to store program instructions executable by the one or more processors to implement:
storing data in the shared storage system on behalf of a plurality of subscribers;
receiving input specifying one or more service level parameter values, each parameter value specifying a quantity of a respective resource that can be allocated for use in executing one or more queries in the shared storage system;
receiving a plurality of queries directed to the data;
and
executing the plurality of queries, wherein to execute the plurality of queries, the program instructions are further executable to implement:
monitoring resource utilization, wherein the monitored resource utilization is resource utilization by the plurality of queries of resources for which service level parameter values have been specified, wherein said monitoring is performed during execution of the plurality of queries;
determining whether the resource utilization by the plurality of queries is consistent with the one or more specified service level parameter values;
and
in response to determining that the resource utilization by one or more of the plurality of queries is not consistent with the one or more specified service level parameter values, attempting to correct the determined inconsistency, wherein said attempting comprises applying a reinforcement learning technique in conjunction with one or more domain-specific constraints to automatically update an execution parameter of at least one of the plurality of queries such that a result of updating the execution parameter is consistent with the one or more domain-specific constraints, wherein an execution parameter is a modifiable parameter affecting execution of the plurality of queries, and wherein the update is performed during execution of the plurality of queries.

10 more elements

A non-transitory, computer-readable storage medium storing program instructions computer-executable to implement:

storing data on behalf of a plurality of subscribers in a shared storage system;
receiving input specifying one or more service level parameter values, each parameter value specifying a quantity of a respective resource that can be allocated for use in executing one or more queries in the shared storage system;
receiving a plurality of queries directed to the data;
and
executing the plurality of queries, wherein to execute the plurality of queries, the program instructions are further computer-executable to implement:
monitoring resource utilization, wherein the monitored resource utilization is resource utilization by the plurality of queries of resources for which service level parameter values have been specified, wherein said monitoring is performed during execution of the plurality of queries;
determining whether the resource utilization by the plurality of queries is consistent with the one or more specified service level parameter values;
and
in response to determining that the resource utilization by one or more of the plurality of queries is not consistent with the one or more specified service level parameter values, attempting to correct the determined inconsistency, wherein said attempting comprises applying a reinforcement learning technique in conjunction with one or more domain-specific constraints to automatically update an execution parameter of at least one of the plurality of queries such that a result of updating the execution parameter is consistent with the one or more domain-specific constraints, wherein an execution parameter is a modifiable parameter affecting execution of the plurality of queries, and wherein the update is performed during execution of the plurality of queries.

6 more elements

A computer-implemented method for providing computing services on behalf of a plurality of subscribers in a shared computing system, comprising:

receiving input specifying one or more service level parameter values, each parameter value specifying a quantity of a respective resource of the shared computing system that can be allocated to a given one of the plurality of subscribers for use in executing one or more computing requests;
receiving a plurality of computing requests;
and
executing the plurality of computing requests, wherein said executing comprises:
monitoring resource utilization, wherein the monitored resource utilization is resource utilization by the plurality of computing requests of resources for which service level parameter values have been specified, wherein said monitoring is performed during execution of the plurality of computing requests;
determining whether the resource utilization by the plurality of computing requests is consistent with the one or more specified service level parameter values;
in response to determining that the resource utilization by one or more of the plurality of computing requests is not consistent with the one or more specified service level parameter values, attempting to correct the determined inconsistency, wherein said attempting comprises automatically updating an execution parameter of at least one of the plurality of computing requests, wherein an execution parameter is a modifiable parameter affecting execution of the plurality of computing requests, and wherein said updating is performed during execution of the plurality of computing requests;
wherein said updating comprises selecting an action from among a plurality of actions dependent on:
previously determined reward values associated with each of the plurality of actions;
and
application of a reinforcement learning technique in conjunction with one or more domain-specific constraints;
wherein each of the plurality of actions comprises a respective specified update to a given execution parameter;
and
wherein selection of the action by the reinforcement learning technique is biased toward actions that are consistent with the one or more domain-specific constraints or is overridden to avoid inconsistency with the one or more one or more domain-specific constraints.

11 more elements

Dependencies are inferred from the claim text.