# US 9,438,482 B2 · Forward request queuing in a distributed edge processing environment > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,438,482 B2 - Document ID: US9438482B2 - Status: Granted - Kind: B2 - Application number: 13/862,695 - Filed: 2013-04-15 - Published: 2013-09-05 - Granted: 2016-09-06 - Earliest claimed: 2003-04-28 - Assignees: Akamai Technologies, Inc. - Applicants: Akamai Technologies, Inc. - Inventors: William E. Weihl; Gene Shekhtman - Web page: https://patentgrep.com/patent/US9438482B2 ## CPC - H04L 65/1086: {session scope modification} - G06F 9/505: {considering the load} - H04L 41/50: Network service management, e.g. ensuring proper service fulfilment according to agreements - H04L 65/611: for multicast or broadcast (systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06) - H04L 65/762: {at the source (reformatting of additional data in video distribution servers H04N21/2355)} - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A server in a distributed environment includes a process that manages incoming client requests and selectively forwards service requests to other servers in the network. The server includes storage in which at least one forwarding queue is established. The server includes code for aggregating service requests in the forwarding queue and then selectively releasing the requests, or some of them, to another server. The queuing mechanism preferably is managed by metadata, which, for example, controls how many service requests may be placed in the queue, how long a given service request may remain in the queue, what action to take in response to a client request if the forwarding queue's capacity is reached, etc. In one embodiment, the server generates an estimate of a current load on an origin server (to which it is sending forwarding requests) and instantiates the forward request queuing when that current load is reached. ## Claims ### Claim 1 Apparatus for use in a distributed network, comprising: at least one hardware processor, an operating system, and a process executed by the hardware processor that manages incoming client requests and selectively forwards given service requests to at least one other server in the distributed network; storage coupled to the processor and including at least one queue; and code tangibly embodied on computer media and executed by the hardware processor, the code comprising: first code operative to make a first determination whether a request should be sent to the other server in the distributed network and, conditioned upon the first determination being made, to make a second determination whether the request should be sent to the queue rather than sent forward to the other server, thereby overriding the first determination, wherein the second determination regarding whether the request should be sent to the queue rather than sent forward to the other server is based on one of: a value of an estimated load on the other server, and a value representing a maximum number of allowed concurrent connections to the other server; second code operative upon determining that the request should be sent forward to the other server to modify the estimated load value and initiate forwarding of the request; and third code operative upon determining that the request should be sent to the queue to insert the request into the queue based on one of: a priority, and an arrival time. ### Claim 2 (depends on 1) The apparatus as described in claim 1, wherein the process is controlled by metadata. ### Claim 3 (depends on 2) The apparatus as described in claim 2, wherein the metadata identifies a maximum number of service requests that may be aggregated in the queue. ### Claim 4 (depends on 2) The apparatus as described in claim 2, wherein the metadata identifies a maximum time that a given service request may remain in the queue. ### Claim 5 (depends on 2) The apparatus as described in claim 2, wherein the metadata identifies a number of given service requests that may be released from the queue at a given time. ### Claim 6 (depends on 2) The apparatus as described in claim 2, wherein the process aggregates in the queue one or more service requests when a rate of incoming client requests to the process exceeds a limit. ### Claim 7 (depends on 6) The apparatus as described in claim 6, wherein the limit is set by metadata. ### Claim 8 (depends on 6) The apparatus as described in claim 6, wherein the limit is a maximum number of service requests that can be forwarded to the given other server at a given time. ### Claim 9 (depends on 1) The apparatus as described in claim 1, further including code responsive to a third determination that a maximum number of service requests are in the queue for taking a given action. ### Claim 10 (depends on 1) The apparatus as described in claim 1, wherein the given action is issuing an error code in response to a given client request being handled by the process. ### Claim 11 (depends on 1) The apparatus as described in claim 1, wherein the given action is issuing given content in response to a given client request being handled by the process. ### Claim 12 (depends on 1) The apparatus as described in claim 1 wherein the code adjusts the estimated load and thus the rate at which the process forwards the service requests by a number of failures or slow connections encountered by the process under a current estimated load.