# US 2023/0057068 A1 · Request throttling using pi-es controller > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2023/0057068 A1 - Document ID: US20230057068A1 - Status: Published application - Kind: A1 - Application number: 17/407,910 - Filed: 2021-08-20 - Published: 2023-02-23 - Earliest claimed: 2021-08-20 - Assignees: Oracle International Corporation - Applicants: Oracle International Corporation - Inventors: Raghuveer Murari Bhandarkar; Ravijeet Ranjit Kumar - Web page: https://patentgrep.com/patent/US20230057068A1 ## CPC - H04L 47/6225: {Fixed service order, e.g. Round Robin} - H04L 43/16: Threshold monitoring - H04L 47/562: {Attaching a time tag to queues} - H04L 47/6255: {queue load conditions, e.g. longest queue first} - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract Techniques for providing request throttling using proportional, integral, and exponential smoothing algorithms are disclosed. A distributed computing system can include a throttler engine that receives a plurality of requests targeting a software component within the distributed computing system. The throttler engine can aggregate the requests into a queue based on a time window. The throttler engine can determine a received request rate and a request rate limit for the software component and then compute a throttled request rate. The throttled request rate can include correction terms derived from proportional and integral computations and a correction term obtained from an exponential smoothing algorithm. The throttler engine can then provide throttled requests from the queue to the software component. ## Claims ### Claim 1 A computer-implemented method, comprising: receiving, at a throttler engine of a distributed computing system, a plurality of requests to use a software component, each request of the plurality of requests associated with a request time; aggregating, by the throttler engine, a subset of the plurality of requests based at least in part on a time window, the request times of each of the requests in the subset corresponding to the time window, and the subset aggregated into a queue; determining, based at least in part on the subset of the plurality of requests, a received request rate; identifying a request rate limit associated with the software component, the request rate limit corresponding to the time window; computing, based at least in part on the received request rate and the request rate limit, a plurality of correction values; determining a throttled request rate based at least in part on the plurality of correction values; identifying one or more throttled requests from the subset, the one or more throttled requests corresponding to the throttled request rate; and providing the one or more throttled requests to the software component. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, further comprising: aggregating a second subset of the plurality of requests based at least in part on a second time window, the request times of each of the requests in the second subset corresponding to the second time window, the second subset aggregated into the queue; determining, based at least in part on the second subset, a second received request rate corresponding to the second time window; updating, based at least in part on the second received request rate, the plurality of correction values; updating the throttled request rate based at least in part on the updated plurality of correction values; identifying additional throttled requests from the second subset, the additional throttled requests corresponding to the updated throttled request rate; and providing the additional throttled requests to the software component. ### Claim 3 (depends on 2) The computer-implemented method of claim 2, wherein computing the plurality of correction values comprises: computing a proportional correction value based at least in part on a first difference between the received request rate and the request rate limit, the proportional correction value comprising a proportional weight; computing an integral correction value based at least in part on the first difference and a second difference between the second received request rate and the request rate limit, the integral correction value comprising an integral weight; and computing a smoothed correction value based at least in part on the first difference and the second difference, the smoothed correction value comprising a smoothed weight. ### Claim 4 (depends on 2) The computer-implemented method of claim 2, further comprising: tuning a plurality of weights based at least in part on a predicted request rate, the plurality of weights associated with the plurality of correction values; detecting a drift in a deviation between the predicted request rate and the received request rate and a subsequent deviation between the predicted request rate and the second received request rate; responsive to detecting the drift exceeding a threshold, re-tuning the plurality of weights based at least in part on a new predicted request rate. ### Claim 5 (depends on 4) The computer-implemented method of claim 4, wherein tuning the plurality of weights comprises hyperparameter tuning. ### Claim 6 (depends on 4) The computer-implemented method of claim 4, further comprising: determining the new predicted request rate based at least in part on the subsequent deviation. ### Claim 7 (depends on 4) The computer-implemented method of claim 4, wherein the plurality of weights comprises predetermined values. ### Claim 8 (depends on 1) The computer-implemented method of claim 1, wherein the queue comprises a first-in first-out queue. ### Claim 9 (depends on 1) The computer-implemented method of claim 1, wherein the request rate limit comprises a static request rate limit. ### Claim 10 A distributed computing system comprising: one or more processors; and one or more memories storing computer-executable instructions that, when executed with the one or more processors, cause the distributed computing system to: receive, at a throttler engine of the distributed computing system, a plurality of requests to use a software component, each request of the plurality of requests associated with a request time; aggregate, by the throttler engine, a subset of the plurality of requests based at least in part on a time window, the request times of each of the requests in the subset corresponding to the time window, and the subset aggregated into a queue; determine, based at least in part on the subset of the plurality of requests, a received request rate; identify a request rate limit associated with the software component, the request rate limit corresponding to the time window; compute, based at least in part on the received request rate and the request rate limit, a plurality of correction values; determine a throttled request rate based at least in part on the plurality of correction values; identify one or more throttled requests from the subset, wherein the one or more throttled requests correspond to the throttled request rate; and provide the one or more throttled requests to the software component. ### Claim 11 (depends on 10) The distributed computing system of claim 10, wherein executing the computer-executable instructions further causes the distributed computing system to: aggregate a second subset of the plurality of requests based at least in part on a second time window, the request times of each of the requests in the second subset corresponding to the second time window, the second subset aggregated into the queue; determine, based at least in part on the second subset, a second received request rate corresponding to the second time window; update, based at least in part on the second received request rate, the plurality of correction values; update the throttled request rate based at least in part on the updated plurality of correction values; identify additional throttled requests from the second subset, the additional throttled requests corresponding to the updated throttled request rate; and provide the additional throttled requests to the software component. ### Claim 12 (depends on 11) The distributed computing system of claim 11, wherein computing the plurality of correction values comprises: computing a proportional correction value based at least in part on a first difference between the received request rate and the request rate limit, the proportional correction value comprising a proportional weight; computing an integral correction value based at least in part on the first difference and a second difference between the second received request rate and the request rate limit, the integral correction value comprising an integral weight; and computing a smoothed correction value based at least in part on the first difference and the second difference, the smoothed correction value comprising a smoothed weight. ### Claim 13 (depends on 11) The distributed computing system of claim 11, wherein executing the computer-executable instructions further causes the distributed computing system to: tune a plurality of weights based at least in part on a predicted request rate, the plurality of weights associated with the plurality of correction values; detect a drift in a deviation between the predicted request rate and the received request rate and a subsequent deviation between the predicted request rate and the second received request rate; and responsive to detecting the drift exceeding a threshold, re-tune the plurality of weights based at least in part on a new predicted request rate. ### Claim 14 (depends on 13) The distributed computing system of claim 13, wherein tuning the plurality of weights comprises hyperparameter tuning. ### Claim 15 (depends on 13) The distributed computing system of claim 13, wherein executing the computer-executable instructions further causes the distributed computing system to: determine the new predicted request rate based at least in part on the subsequent deviation. ### Claim 16 A computer readable medium storing computer-executable instructions that, when executed by one or more processors, cause a distributed computing system to at least: receive, at a throttler engine of the distributed computing system, a plurality of requests to use a software component, each request of the plurality of requests associated with a request time; aggregate, by the throttler engine, a subset of the plurality of requests based at least in part on a time window, the request times of each of the requests in the subset corresponding to the time window, and the subset aggregated into a queue; determine, based at least in part on the subset of the plurality of requests, a received request rate; identify a request rate limit associated with the software component, the request rate limit corresponding to the time window; compute, based at least in part on the received request rate and the request rate limit, a plurality of correction values; determine a throttled request rate based at least in part on the plurality of correction values; identify one or more throttled requests from the subset, wherein the one or more throttled requests correspond to the throttled request rate; and provide the one or more throttled requests to the software component. ### Claim 17 (depends on 16) The computer readable medium of claim 16, wherein executing the computer-executable instructions further causes the distributed computing system to: aggregate a second subset of the plurality of requests based at least in part on a second time window, the request times of each of the requests in the second subset corresponding to the second time window, the second subset aggregated into the queue; determine, based at least in part on the second subset, a second received request rate corresponding to the second time window; update, based at least in part on the second received request rate, the plurality of correction values; update the throttled request rate based at least in part on the updated plurality of correction values; identify additional throttled requests from the second subset, the additional throttled requests corresponding to the updated throttled request rate; and provide the additional throttled requests to the software component. ### Claim 18 (depends on 17) The computer readable medium of claim 17, wherein computing the plurality of correction values comprises: computing a proportional correction value based at least in part on a first difference between the received request rate and the request rate limit, the proportional correction value comprising a proportional weight; computing an integral correction value based at least in part on the first difference and a second difference between the second received request rate and the request rate limit, the integral correction value comprising an integral weight; and computing a smoothed correction value based at least in part on the first difference and the second difference, the smoothed correction value comprising a smoothed weight. ### Claim 19 (depends on 17) The computer readable medium of claim 17, wherein executing the computer-executable instructions further causes the distributed computing system to: tune a plurality of weights based at least in part on a predicted request rate, the plurality of weights associated with the plurality of correction values; detect a drift in a deviation between the predicted request rate and the received request rate and a subsequent deviation between the predicted request rate and the second received request rate; and responsive to detecting the drift exceeding a threshold, re-tune the plurality of weights based at least in part on a new predicted request rate. ### Claim 20 (depends on 19) The computer readable medium of claim 19, wherein tuning the plurality of weights comprises hyperparameter tuning.