US 2018/0295044 A1 · AUTOMATED MANAGEMENT OF SERVICE INSTANCES

Abstract

A system, apparatus, and methods are provided for automatically managing a collection of service instances. A selected or random instance of the service isredline-tested to determine a maximum level of sustained and stable performance (e.g., a maximum or approximate maximum load or throughput of the service instance). This redline value may represent a highest service request rate (e.g., in queries per second or qps) that should be delivered to some or all instances of the service. An expected range of demand for the service is then estimated for a future time period, possibly from analysis and/or observations of past demand, and a maximum and minimum number of instances for handling that range of demand, at an acceptable level of performance, can be determined. During the time period, additional instances may be created and superfluous instances may be removed automatically, based on actual demand.

Claims20

Dependent claims

A computer-implemented method of managing a collection of instances of a service, the method comprising:

determining a maximum request rate at which a selected instance of the service provides a stable level of performance;
estimating a range of request rates to be received by the service during a future period of time;
and
during the period of time, operating a computer to:
when an average request rate at the service instances is above a first threshold, automatically create an additional instance of the service;
and
when the average request rate at the service instances is below a second threshold, automatically remove at least one instance of the service.

4 more elements

An apparatus, comprising:

at least one processor;
and
memory storing instructions that, when executed by the at least one processor, cause the apparatus to:
determine a maximum request rate at which a selected instance of the service provides a stable level of performance;
estimate a range of request rates to be received by the service during a future period of time;
and
during the period of time:
when an average request rate at the service instances is above a first threshold, automatically create an additional instance of the service;
and
when the average request rate at the service instances is below a second threshold, automatically remove at least one instance of the service.

7 more elements

A system for managing a collection of instances of a service, comprising:

one or more processors;
a redline module comprising a non-transitory computer readable medium storing instructions that, when executed, cause the system to determine a maximum request rate at which a selected instance of the service provides a stable level of performance;
and
a fleet management module comprising a non-transitory computer readable medium storing instructions that, when executed, cause the system to:
estimate a range of request rates to be received by the service during a future period of time;
and
during the period of time:
when an average request rate at the service instances is above a first threshold, automatically create an additional instance of the service;
and
when the average request rate at the service instances is below a second threshold, automatically remove at least one instance of the service.

7 more elements

Dependencies are inferred from the claim text.

Automated management of service instances

Life of this patent

Automated management of service instances

Life of this patent

Abstract

Claims20