# US 2025/0168117 A1 · Normalized concurrency limits for throttling and fault isolation in a routing service

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0168117 A1
- Document ID: US20250168117A1
- Status: Published application
- Kind: A1
- Application number: 18/514,339
- Filed: 2023-11-20
- Published: 2025-05-22
- Earliest claimed: 2023-11-20
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Kushal Suresh NARKHEDE; Daniel WALZENBACH
- Web page: https://patentgrep.com/patent/US20250168117A1

## CPC

- H04L 47/12: Avoiding congestion; Recovering from congestion
- H04L 47/25: with rate being modified by the source upon detecting a change of network conditions
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

The processing capabilities of the machines in a routing service are evaluated and traffic patterns indicative of how calls are made to different dependency services are identified. The capabilities of the routing service and the traffic patterns are used to generate a dynamic limit model that dynamically limits the number of calls made to each dependency service. When the capabilities of the routing service change, the dynamic limit model automatically adjusts the limit corresponding to each dependency service.

## Claims

### Claim 1

A computer implemented method, comprising:
accessing capability data indicative of request routing volume capabilities of a request routing service that routes requests to backend servers, the request routing service, when routing a request, calling a dependency service, of a plurality of different dependency services, to identify a destination of a target backend service corresponding to the request;
accessing traffic pattern data indicative of a level of usage of each of the plurality of dependency services;
generating a separate dynamic limit model corresponding to each of the plurality of dependency services, each of the separate dynamic limit models being configured to generate a dynamic concurrency limit indicative of a number of pending calls that are permitted for the corresponding dependency service; and
sending the dynamic limit models to a throttling system that controls throttling of calls to each of the plurality of dependency services.

### Claim 2 (depends on 1)

The computer implemented method of claim 1 wherein accessing capability data comprises:
identifying a number of requests (M) per unit time that can be processed by a portion of the request routing service based on machine identifying data and based on the request routing volume capabilities.

### Claim 3 (depends on 2)

The computer implemented method of claim 2 wherein accessing traffic pattern data comprises:
identifying a number of calls (N) to each of the dependency services, per unit time.

### Claim 4 (depends on 3)

The computer implemented method of claim 3 wherein generating a separate dynamic limit model for each of the plurality of dependency services comprises:
selecting a dependency service of the plurality of dependency services; and
generating, as a dynamic limit model for the selected dependency service, a model based on a ratio of N/M.

### Claim 5 (depends on 4)

The computer implemented method of claim 4 and further comprising:
applying the dynamic limit model for the selected dependency service to obtain a dynamic limit for the dependency service.

### Claim 6 (depends on 5)

The computer implemented method of claim 5 wherein applying the dynamic limit model for the selected dependency service comprises:
calculating a dynamic limit for the selected dependency service based on the M and the ratio.

### Claim 7 (depends on 5)

The computer implemented method of claim 5 and further comprising:
detecting a change in the number of requests M that can be processed to obtain an updated value of M; and
applying the dynamic limit model to the updated value of M for the selected dependency service to obtain an updated dynamic limit for the selected dependency service.

### Claim 8 (depends on 7)

The computer implemented method of claim 7 and further comprising:
applying the dynamic limit model corresponding to each dependency service to the updated value of M to obtain an updated dynamic limit for each corresponding dependency service.

### Claim 9

A computer implemented method, comprising:
configuring a throttling system in a routing service, to obtain a configured throttling system, by obtaining, at the throttling system in a routing service, a plurality of dynamic limit models, one dynamic limit model corresponding to each of a plurality of different dependency services, and running the plurality of dynamic limit models to obtain a different dynamic concurrency limit corresponding to each of the plurality of different dependency services; and
performing throttling using the configured throttling system by performing steps comprising:
receiving, at a request routing system, a request to perform an operation at a target backend server, of a plurality of different backend servers;
generating a call to an identified dependency service, of the plurality of different dependency services, to identify a request destination corresponding to the target backend server;
determining whether the call meets the dynamic concurrency limit corresponding to the identified dependency service; and
if the call meets the dynamic concurrency limit corresponding to the identified dependency service, then rejecting the call; and
if not, enqueueing the call for the identified dependency service.

### Claim 10 (depends on 9)

The computer implemented method of claim 9 and further comprising:
generating each of the plurality of dynamic limit models based on capabilities of machines used to implement the routing service.

### Claim 11 (depends on 10)

The computer implemented method of claim 10 wherein generating each of the plurality of dynamic limit models comprises:
obtaining traffic pattern data indicative of a proportion of times that a call is generated to each of the plurality of dependency services; and
generating each of the plurality of dynamic limit models based on the traffic pattern data.

### Claim 12 (depends on 11)

The computer implemented method of claim 11 wherein running the plurality of dynamic limit models comprises:
intermittently evaluating the capabilities of the machines to obtain updated capability data; and
running the plurality of dynamic limit models based on the updated capability data to obtain an updated dynamic concurrency limit corresponding to each of the plurality of different dependency service.

### Claim 13 (depends on 11)

The computer implemented method of claim 11 wherein obtaining traffic pattern data comprises:
intermittently evaluating traffic patterns at different locations of the routing service to obtain traffic pattern data for each of the different locations.

### Claim 14 (depends on 13)

The computer implemented method of claim 13 wherein generating each of the plurality of dynamic limit models comprises:
generating a different dynamic limit model for each of the plurality if different dependency services at each different location based on the traffic pattern data corresponding to the location.

### Claim 15 (depends on 12)

The computer implemented method of claim 12 wherein intermittently evaluating traffic patterns comprises:
counting a number of calls to each of the plurality of different dependency services over a time period.

### Claim 16 (depends on 15)

The computer implemented method of claim 15 wherein intermittently evaluating traffic patterns comprises:
for each of the plurality of different dependency services, calculating the count of the number of calls to the dependency service relative to a total number of calls to all of the plurality of dependency services over the time period.

### Claim 17

A computer system, comprising:
at least one processor; and
memory storing computer executable instructions which, when executed by the at least one processor, cause the at least one processor to perform steps, comprising:
generating a separate dynamic concurrency limit model corresponding to each of a plurality of dependency services called by a request routing service to identify a destination of a target backend service corresponding to a request; and
sending the dynamic limit models to a throttling system that controls throttling of calls to each of the plurality of dependency services.

### Claim 18 (depends on 17)

The computer system of claim 17 wherein the steps further comprise:
accessing capability data indicative of request routing volume capabilities of the request routing system; and
accessing traffic pattern data indicative of a level of usage of each of the plurality of dependency services.

### Claim 19 (depends on 18)

The computer system of claim 18 wherein generating a separate dynamic concurrency limit model comprises:
generating the separate dynamic concurrency limit models based on the capability data and the traffic pattern data.

### Claim 20 (depends on 19)

The computer system of claim 19 wherein the steps further comprise:
running the plurality of dynamic limit models to obtain a different dynamic concurrency limit corresponding to each of the plurality of dependency services.
