# US 12,217,088 B2 · Autoscaling and throttling in an elastic cloud service > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,217,088 B2 - Document ID: US12217088B2 - Status: Granted - Kind: B2 - Application number: 18/497,260 - Filed: 2023-10-30 - Published: 2024-02-22 - Granted: 2025-02-04 - Earliest claimed: 2021-06-23 - Assignees: Snowflake Inc. - Applicants: Snowflake Inc. - Inventors: Johan Harjono; Daniel Geoffrey Karp; Kunal Prafulla Nabar; Rares Radut; Arthur Kelvin Shi - Web page: https://patentgrep.com/patent/US12217088B2 ## CPC - G06F 9/5005: {to service a request} - G06F 9/3555: {using scaling, e.g. multiplication of index} - G06F 9/505: {considering the load} - G06F 9/5077: {Logical partitioning of resources; Management or configuration of virtualized resources (specific details on emulation or internal functioning of virtual machines G06F9/455)} - G06F 9/5083: {Techniques for rebalancing the load in a distributed system} - G06F 2209/504: Resource capping - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Techniques described herein can optimize usage of computing resources in a data system. Dynamic throttling can be performed locally on a computing resource in the foreground and autoscaling can be performed in a centralized fashion in the background. Dynamic throttling can lower the load without overshooting while minimizing oscillation and reducing the throttle quickly. Autoscaling may involve scaling in or out the number of computing resources in a cluster as well as scaling up or down the type of computing resources to handle different types of situations. ## Claims ### Claim 1 A method comprising: monitoring, by a computing resource assigned to a cluster of a plurality of clusters, resource usage at the respective computing resource; throttling a number of new inbound requests received by the respective computing resource based on the monitored resource usage, the throttling comprising: determining that the resource usage has exceeded a maximum threshold; generating an estimate of an allotment of new inbound requests to be received by the respective computing resource to lower the resource usage below the maximum threshold; and setting a gateway limit of new inbound requests based on the estimate to throttle new inbound requests received by the respective computing resource; transmitting information and statistics of usage levels to a central sever, the central server being configured to adjust a number of computing resources assigned to each of the plurality of clusters based on the information and statistics; and receiving instructions from the central server regarding cluster assignment. ### Claim 2 (depends on 1) The method of claim 1, wherein the throttling includes: for a defined time interval, periodically adjusting the gateway limit based on a current resource usage. ### Claim 3 (depends on 2) The method of claim 2, wherein the estimate of the allotment is a first estimate and wherein the defined time interval is a first defined time interval, the method further comprising: after the defined time interval, generating a second estimate of the allotment of new inbound requests to be received by the computing resource to lower the resource usage below the maximum threshold; setting the gateway limit based on the second estimate to throttle new inbound requests received by the computing resource; for a second defined time interval, periodically adjusting the gateway limit based on the current resource usage; and after the second defined time interval, implementing a full recovery of the gateway limit. ### Claim 4 (depends on 2) The method of claim 2, wherein the gateway limit is defined on a per-account basis. ### Claim 5 (depends on 2) The method of claim 2, further comprising: rejecting new inbound requests over the gateway limit. ### Claim 6 (depends on 5) The method of claim 5, further comprising: transmitting an instruction to a sender of a rejected inbound request to re-send the request after a set time. ### Claim 7 (depends on 1) The method of claim 1, wherein the central server is configured to generate a plurality of moving average windows of usage levels of the computing resources in each of the plurality of clusters, wherein the adjusting the number of computing resources assigned to at least one of the plurality of clusters is based on the plurality of moving average windows. ### Claim 8 (depends on 7) The method of claim 7, wherein in an event the number of computer resources for a respective cluster is increased, the increase of the number of computing resource for the respective cluster is based on a single window of the plurality of moving average windows. ### Claim 9 (depends on 7) The method of claim 7, wherein in an event the number of computer resources for a respective cluster is decreased, the decrease of the number of computing resource for the respective cluster is based on at least two of the plurality of moving average windows. ### Claim 10 (depends on 1) The method of claim 1, wherein the throttling further includes: determining that the resource usage is below a minimum threshold, determining that a rejection rate of new inbound requests is above a rejection threshold, and expanding an allotment of new inbound requests to be received by the computing resource. ### Claim 11 A system comprising: one or more processors of a machine; and at least one memory storing instructions that, when executed by the one or more processors, cause the machine to perform operations comprising: monitoring, by a computing resource assigned to a cluster of a plurality of clusters, resource usage at the respective computing resource; throttling a number of new inbound requests received by the respective computing resource based on the monitored resource usage, the throttling comprising: determining that the resource usage has exceeded a maximum threshold; generating an estimate of an allotment of new inbound requests to be received by the respective computing resource to lower the resource usage below the maximum threshold; and setting a gateway limit of new inbound requests based on the estimate to throttle new inbound requests received by the respective computing resource; transmitting information and statistics of usage levels to a central sever, the central server being configured to adjust a number of computing resources assigned to each of the plurality of clusters based on the information and statistics; and receiving instructions from the central server regarding cluster assignment. ### Claim 12 (depends on 11) The system of claim 11, wherein the throttling includes: for a defined time interval, periodically adjusting the gateway limit based on a current resource usage. ### Claim 13 (depends on 12) The system of claim 12, wherein the estimate of the allotment is a first estimate and wherein the defined time interval is a first defined time interval, the operations further comprising: after the defined time interval, generating a second estimate of the allotment of new inbound requests to be received by the computing resource to lower the resource usage below the maximum threshold; setting the gateway limit based on the second estimate to throttle new inbound requests received by the computing resource; for a second defined time interval, periodically adjusting the gateway limit based on the current resource usage; and after the second defined time interval, implementing a full recovery of the gateway limit. ### Claim 14 (depends on 12) The system of claim 12, wherein the gateway limit is defined on a per-account basis. ### Claim 15 (depends on 12) The system of claim 12, further comprising: rejecting new inbound requests over the gateway limit. ### Claim 16 (depends on 15) The system of claim 15, further comprising: transmitting an instruction to a sender of a rejected inbound request to re-send the request after a set time. ### Claim 17 (depends on 11) The system of claim 11, wherein the central server is configured to generate a plurality of moving average windows of usage levels of the computing resources in each of the plurality of clusters, wherein the adjusting the number of computing resources assigned to at least one of the plurality of clusters is based on the plurality of moving average windows. ### Claim 18 (depends on 17) The system of claim 17, wherein in an event the number of computer resources for a respective cluster is increased, the increase of the number of computing resource for the respective cluster is based on a single window of the plurality of moving average windows. ### Claim 19 (depends on 17) The system of claim 17, wherein in an event the number of computer resources for a respective cluster is decreased, the decrease of the number of computing resource for the respective cluster is based on at least two of the plurality of moving average windows. ### Claim 20 (depends on 11) The system of claim 11, wherein the throttling further includes: determining that the resource usage is below a minimum threshold, determining that a rejection rate of new inbound requests is above a rejection threshold, and expanding an allotment of new inbound requests to be received by the computing resource. ### Claim 21 A machine-storage medium embodying instructions that, when executed by at least one machine, cause the at least one machine to perform operations comprising: monitoring, by a computing resource assigned to a cluster of a plurality of clusters, resource usage at the respective computing resource; throttling a number of new inbound requests received by the respective computing resource based on the monitored resource usage, the throttling comprising: determining that the resource usage has exceeded a maximum threshold; generating an estimate of an allotment of new inbound requests to be received by the respective computing resource to lower the resource usage below the maximum threshold; and setting a gateway limit of new inbound requests based on the estimate to throttle new inbound requests received by the respective computing resource; transmitting information and statistics of usage levels to a central sever, the central server being configured to adjust a number of computing resources assigned to each of the plurality of clusters based on the information and statistics; and receiving instructions from the central server regarding cluster assignment. ### Claim 22 (depends on 21) The machine-storage medium of claim 21, wherein the throttling includes: for a defined time interval, periodically adjusting the gateway limit based on a current resource usage. ### Claim 23 (depends on 22) The machine-storage medium of claim 22, wherein the estimate of the allotment is a first estimate and wherein the defined time interval is a first defined time interval, the operations further comprising: after the defined time interval, generating a second estimate of the allotment of new inbound requests to be received by the computing resource to lower the resource usage below the maximum threshold; setting the gateway limit based on the second estimate to throttle new inbound requests received by the computing resource; for a second defined time interval, periodically adjusting the gateway limit based on the current resource usage; and after the second defined time interval, implementing a full recovery of the gateway limit. ### Claim 24 (depends on 22) The machine-storage medium of claim 22, wherein the gateway limit is defined on a per-account basis. ### Claim 25 (depends on 22) The machine-storage medium of claim 22, further comprising: rejecting new inbound requests over the gateway limit. ### Claim 26 (depends on 25) The machine-storage medium of claim 25, further comprising: transmitting an instruction to a sender of a rejected inbound request to re-send the request after a set time. ### Claim 27 (depends on 21) The machine-storage medium of claim 21, wherein the central server is configured to generate a plurality of moving average windows of usage levels of the computing resources in each of the plurality of clusters, wherein the adjusting the number of computing resources assigned to at least one of the plurality of clusters is based on the plurality of moving average windows. ### Claim 28 (depends on 27) The machine-storage medium of claim 27, wherein in an event the number of computer resources for a respective cluster is increased, the increase of the number of computing resource for the respective cluster is based on a single window of the plurality of moving average windows. ### Claim 29 (depends on 27) The machine-storage medium of claim 27, wherein in an event the number of computer resources for a respective cluster is decreased, the decrease of the number of computing resource for the respective cluster is based on at least two of the plurality of moving average windows. ### Claim 30 (depends on 21) The machine-storage medium of claim 21, wherein the throttling further includes: determining that the resource usage is below a minimum threshold, determining that a rejection rate of new inbound requests is above a rejection threshold, and expanding an allotment of new inbound requests to be received by the computing resource.