Minimal configuration and priority-based frugal throttling solution
AssigneeAmazon Technologies, Inc.
InventorsDeepak Aggarwal, Monishkumar Gajendran
Life of this patent
Estimating the term…
Abstract
A computing system may detect that a service, which receives a plurality of task requests associated with clients or profiles, is under duress based on performance information associated with the service. The computing system, responsive to detecting that the service is under duress, may successively, until the service is detected to not be under duress, select a profile based on a respective volume of requests associated with the profile, apply a task request limit to the profile, and detect whether the service is still under duress.
Claims20
A computer-implemented method, comprising:
- receiving a plurality of task requests at a service from a plurality of clients, the service being a computer system to which task requests are submittable over a network;
- obtaining performance information of the service and a downstream service;
- detecting that the downstream service is throttling requests, received from the service, at a first throttling rate based on a number of request denials received by the service from the downstream service in a period of time;
- using the obtained performance information and based on determining that the first throttle rate is above a failure throttle rate threshold, determining that the service is under duress;
- responsive to determining that the service is under duress, successively performing the following operations multiple times until the service is determined to not be under duress, wherein at least two times of the multiple times that the following operations are performed, a different subset of clients is selected from the plurality of clients:
- selecting a subset of clients from the plurality of clients based on a respective volume of requests associated with individual clients of the subset of clients;
- applying a throttle to the selected subset of clients;
- and
- determining whether the service is still under duress.
A system, comprising at least one computing device configured to implement one or more services, wherein the one or more services:
- detect that a service is under duress based on performance information associated with the service and based on detecting that responses, indicating a failure, from a downstream service responsive to requests received from the service, are received at a first failure rate that is above a failure rate threshold, wherein the service receives a plurality of task requests associated with a plurality of profiles;
- and
- responsive to detecting that the service is under duress, successively perform the following set of operations at least two times, until it is determined that the service is no longer under duress:
- select a subset of profiles from the plurality of profiles based on a respective volume of requests associated with individual profiles of the plurality of profiles, the subset of profiles comprising individual profiles having a highest volume of requests of the plurality of requests, individual profiles of the plurality of profiles associated with a client device;
- apply a task request limit to the subset of profiles;
- and
- detect whether the service is still under duress.
A non-transitory computer-readable storage medium having stored thereon executable instructions that, as a result of being executed by one or more processors of a computer system, cause the computer system to at least:
- detect that a service is under duress based at least in part on detecting that a downstream service is throttling requests received from the service at a first throttle rate that is above a failure throttle rate threshold, wherein the first throttle rate is determined based on a number of request denials received by the service from the downstream service in a period of time, wherein the service receives a plurality of task requests associated with one or more profiles;
- and
- responsive to detecting that the service is under duress, successively perform the following operations multiple times until the service is detected to not be under duress, wherein at least two times of the multiple times that the following operations are performed, a different profile is selected from the one or more profiles:
- select a profile from the one or more profiles having a respective volume of requests associated with the profile above a threshold, the profile associated with a highest request number or request volume of the one or more profiles and associated with a client;
- apply a task request limit to the profile;
- and
- detect whether the service is still under duress.
Dependencies are inferred from the claim text.