Normalized concurrency limits for throttling and fault isolation in a routing service
AssigneeMicrosoft Technology Licensing, LLC
InventorsKushal Suresh NARKHEDE, Daniel WALZENBACH
Life of this patent
Estimating the term…
Abstract
The processing capabilities of the machines in a routing service are evaluated and traffic patterns indicative of how calls are made to different dependency services are identified. The capabilities of the routing service and the traffic patterns are used to generate a dynamic limit model that dynamically limits the number of calls made to each dependency service. When the capabilities of the routing service change, the dynamic limit model automatically adjusts the limit corresponding to each dependency service.
Claims20
A computer implemented method, comprising:
- accessing capability data indicative of request routing volume capabilities of a request routing service that routes requests to backend servers, the request routing service, when routing a request, calling a dependency service, of a plurality of different dependency services, to identify a destination of a target backend service corresponding to the request;
- accessing traffic pattern data indicative of a level of usage of each of the plurality of dependency services;
- generating a separate dynamic limit model corresponding to each of the plurality of dependency services, each of the separate dynamic limit models being configured to generate a dynamic concurrency limit indicative of a number of pending calls that are permitted for the corresponding dependency service;
- and
- sending the dynamic limit models to a throttling system that controls throttling of calls to each of the plurality of dependency services.
A computer implemented method, comprising:
- configuring a throttling system in a routing service, to obtain a configured throttling system, by obtaining, at the throttling system in a routing service, a plurality of dynamic limit models, one dynamic limit model corresponding to each of a plurality of different dependency services, and running the plurality of dynamic limit models to obtain a different dynamic concurrency limit corresponding to each of the plurality of different dependency services;
- and
- performing throttling using the configured throttling system by performing steps comprising:
- receiving, at a request routing system, a request to perform an operation at a target backend server, of a plurality of different backend servers;
- generating a call to an identified dependency service, of the plurality of different dependency services, to identify a request destination corresponding to the target backend server;
- determining whether the call meets the dynamic concurrency limit corresponding to the identified dependency service;
- and
- if the call meets the dynamic concurrency limit corresponding to the identified dependency service, then rejecting the call;
- and
- if not, enqueueing the call for the identified dependency service.
A computer system, comprising:
- at least one processor;
- and
- memory storing computer executable instructions which, when executed by the at least one processor, cause the at least one processor to perform steps, comprising:
- generating a separate dynamic concurrency limit model corresponding to each of a plurality of dependency services called by a request routing service to identify a destination of a target backend service corresponding to a request;
- and
- sending the dynamic limit models to a throttling system that controls throttling of calls to each of the plurality of dependency services.
Dependencies are inferred from the claim text.