Method and system for cluster rate limiting in a cloud computing system

A method of rate limiting clusters of distributed service instances running on distributed compute servers over a network in a cloud computing system, the method comprising:

• receiving, at a rate limit controller, a request to determine a token distribution for each one of a plurality of distributed rate limiters, each rate limiter associated with one of the clusters of distributed service instances, each one of the plurality of distributed rate limiters rate limiting a service instance of a corresponding one of the clusters of distributed service instances;
• determining, at the rate limit controller, the token distribution for each one of the plurality of distributed rate limiters and remaining distribution in the rate limit controller, the token distribution comprising a maximum token number and a token self-generating rate to generate tokens by each distributed rate limiter and generate remaining tokens by the rate limit controller to minimize interactions between the rate limit controller and distributed rate limiters over the network;
• outputting, by the rate limiting controller, the maxim urn token number and the token self-generating rate to each one of the plurality of distributed rate limiters;
• receiving, from at least a first rate limiter of the plurality of distributed rate limiters, at least a first request for additional tokens;
• and
• outputting an adjusted token distribution to the first rate limiter, the adjusted token distribution including at least one of an adjusted maximum token number or an adjusted token self-generating rate based on the first request and based on token consumption information for at least the first rate limiter of the plurality of distributed rate limiters.