Adaptive load balancing for distributed systems
AssigneeAmazon Tehonlogies, Inc.
InventorsHimanshu Jindal, Christoph Saalfeld
Life of this patent
Estimating the term…
Abstract
Adaptive load balancing may be implemented for a distributed system. An evaluation of criteria for creating a mapping for an identifier associated with requests may be performed. The created mapping may route requests to a distributed system to request handling nodes that cache request information for performing the requests. When requests are received, if an identifier determined to be associated with the request is mapped to request handling nodes, then the request may be sent to the identified request handling nodes instead of performing a load balancing technique to select a request handling node for the request.
Claims20
A distributed system, comprising:
- a plurality of request handling nodes, respectively comprising at least one processor and a memory, that perform requests received by the distributed system and cache request information obtained to perform the requests;
- and
- a load balancer, respectively comprising another processor and another memory, that is configured to:
- for a common identifier associated with requests to the distributed system:
- evaluate a performance profile for requests associated with the common identifier with respect to one or more criteria to make a determination to route requests to one or more of the request handling nodes that cache request information to perform the requests associated with the common identifier;
- store a mapping for the common identifier that is utilized to route the requests associated with the common identifier to the one or more request handling nodes, wherein the one or more request handling nodes are identified by the mapping for the common identifier;
- receive a request to be performed by the distributed system;
- determine that the request to be performed by the distributed system is associated with the common identifier;
- and
- forward the request to the one or more request handling nodes.
A method, comprising:
- evaluating, by a load balancer, one or more criteria for determining whether to route requests to one or more request handling nodes of a distributed system that cache request information to perform the requests, wherein the requests are associated with a common identifier, wherein the evaluating the one or more criteria comprises determining whether a performance profile for the requests associated with the common identifier exceeds a threshold criteria for creating a mapping for the common identifier;
- determining to route the requests associated with the common identifier to the one or more request handling nodes according to the evaluation of the one or more criteria;
- responsive to the determining to route the requests, creating, by the load balancer, the mapping for the common identifier that is utilized to route the requests associated with the common identifier to the one or more request handling nodes;
- receiving, by the load balancer, a request;
- determining, by the load balancer, that the request is associated with the common identifier;
- and
- sending, by the load balancer, the request to the one or more request handling nodes identified according to the mapping for the common identifier.
One or more non-transitory, computer-readable storage media, storing program instructions that when executed on or across one or more computing devices cause the one or more computing devices to implement a load balancer that implements:
- evaluating one or more criteria for determining whether to route requests to one or more request handling nodes of a distributed system that cache request information to perform the requests, wherein the requests are associated with a common identifier, wherein the evaluating the one or more criteria comprises determining whether a performance profile associated with the common identifier exceeds a threshold criteria for creating a mapping for the common identifier;
- determining to route the requests associated with the common identifier to the one or more request handling nodes according to the evaluation of the one or more criteria;
- responsive to the determining to route the requests, storing the mapping for the common identifier that is utilized to route the requests associated with the common identifier to the one or more request handling nodes, wherein the one or more request handling nodes are identified by the mapping for the common identifier;
- receiving a request to be performed by the distributed system;
- determining that the request to be performed by the distributed system is associated with the common identifier;
- and
- sending the request to the one or more request handling nodes.
Dependencies are inferred from the claim text.