Adaptive cloud request handling

A system comprising:

• at least one data processor;
• and
• memory storing instructions which, when executed by the at least one data processor, result in operations comprising:
• receiving, by a cloud computing system, a plurality of requests originating from a plurality of tenants;
• dispatching, by a load balancer forming part of the cloud computing system, each request to one of a plurality of groups of servers based on a group mapping table, each group of servers comprising a plurality of servers, each group of servers dispatching each request to one of the plurality of servers within such group of servers;
• and
• providing, by the cloud computing system, responses to the requests to the respective tenants as generated by the respective servers within the respective groups of servers;
• wherein the group mapping table is generated by:
• first monitoring historical request statistics for each of the tenants;
• second monitoring current resource consumption by the plurality of groups of servers when handling requests from the tenants;
• and
• associating each tenant with one of the groups of servers based on the first and second monitoring.

A computer-implemented method comprising:

• first monitoring request statistics associated with a plurality of requests received by a cloud computing system originating from each of a plurality of tenants;
• second monitoring resource consumption by groups of servers forming part of the cloud computing system when handling the plurality of requests received by the cloud computing system;
• generating tenant metrics for each tenant based on the first monitoring;
• generating server group metrics for each group of servers based on the second monitoring;
• generating, based on the generated tenant metrics and the generated server group metrics, a group mapping table configured to optimize a cache hit ratio by the groups of servers;
• receiving, by the cloud computing system, a plurality of new requests originating from a plurality of tenants;
• dispatching, by a load balancer forming part of the cloud computing system, each new request to one of a plurality of groups of servers based on the group mapping table, each group of servers comprising a plurality of servers, each group of servers dispatching each request to one of the plurality of servers within such group of servers;
• and
• providing, by the cloud computing system, responses to the new requests to the respective tenants as generated by the respective servers within the respective groups of servers.