Highly scalable system and method of regulating internet traffic to server farm to support (min,max) bandwidth usage-based service level agreements

A system for controlling and managing Internet server farm traffic through a plurality of servers, the server farm traffic arriving at a server farm as inbound traffic organized by customer (I) and traffic type (j) and leaving the server farm as outbound traffic, the system being operable to control and manage the outbound traffic in accordance with outbound bandwidth usage-based service level agreements of form (Bmin,Bmax) by computing a target rate (Rt) for each customer traffic type (i,j) that supports the outbound bandwidth usage-based service level agreements of form (Bmin,Bmax) and then admitting a portion of the inbound traffic at an admitted rate (Ra) while rejecting at a rejected rate (Rr) a remaining portion of the inbound traffic that if processed would cause the outbound traffic for the customer traffic type (i,j) to exceed the target rate (Rt), the system being further operable to control and manage the outbound traffic in accordance with a server resource manager that establishes an absolute bound (Rbound) of a target rate (Rt) for each customer traffic type (i,j), the system comprising:

• an inbound traffic scheduler device operable to collect the admitted rate (Ra) and the rejected rata (Rr) of inbound traffic for each customer traffic type (i,j), collect the bound (Rbound) from any server resource manager and collect the outbound traffic (B) for each customer traffic type (i,j), the inbound traffic scheduler device being further operable to compute an expected bandwidth usage (b) per request for each customer traffic type (i,j) and use the expected bandwidth usage (b) to compute a target rate (Rt) for inbound traffic for each customer traffic type (i,j) to support the outbound bandwidth usage-based service level agreements of form (Bmin,Bmax);
• an inbound traffic limiter operable to receive the target rate (Rt) from the inbound traffic scheduler device, admit inbound traffic based on the target rate (Rt), track the volume of admitted inbound traffic (Ra) and the volume of rejected inbound traffic (Rr) for each customer traffic type (i,j), and relay the amount of admitted inbound traffic (Ra) and the amount of rejected traffic (Rr) to the inbound traffic scheduler device;
• and
• an inbound traffic dispatching network operable to classify incoming traffic, the inbound traffic dispatching network being controlled by the inbound traffic limiter to selectively dropping packets arriving in the inbound traffic limiter if the inbound traffic exceeds the target rate (Rt), the inbound traffic dispatching network further being comprised of a high-speed LAN with dispatchers to dispatch the admitted inbound traffic (Ra) to the servers.

A method for controlling and managing Internet server farm traffic through a plurality of servers, the server farm traffic arriving at a server farm as inbound traffic organized by customer (I) and traffic type (j) and leaving the server farm as outbound traffic that is controlled and managed in accordance with outbound bandwidth usage-based service level agreements (Bmin(i,j),Bmax(i,j)) by computing a target rate (Rt) for each customer traffic type (i,j) that supports the outbound bandwidth usage-based service level agreements of form (Bmin(i,j),Bmax(i,j) and then admitting a portion of the inbound traffic at an admitted rate (Ra) while rejecting at a rejected rate (Rr) a remaining portion of the inbound traffic that if processed would cause the outbound traffic for the customer traffic type (i,j) to exceed the target rate (Rt), the method comprising the steps of:

• collecting the admitted rate (Ra(i,j)) of inbound traffic for each customer traffic type (i,j);
• collecting the rejected rate (Rr(i,j)) of inbound traffic for each customer traffic type (i,j);
• collecting the outbound traffic (B(i,j)) for each customer traffic type (i,j);
• collecting the absolute bound (Rbound(i,j)) on the target rate (Rt(i,j)) for each customer traffic type (i,j);
• computing an expected bandwidth usage (b(i,j)) per TCP connection request for each customer traffic type (i,j);
• using the expected bandwidth usage (b(i,j)) to compute the target rate (Rt(i,j)) for each customer traffic type (i,j) based on the admitted rate (Ra(i,j)), the rejected rate (Rr(i,j)), the outbound traffic (B(i,j)), the expected bandwidth usage (b(i,j)) and the outbound bandwidth usage-based service level agreements (Bmin(i,j),Bmax(i,j));
• admitting inbound traffic based on the target rate (Rt(i,j)), rejecting inbound traffic that exceeds the target rate (Rt(i,j)) and tracking the volume of admitted inbound traffic (Ra(i,j)) and the volume of rejected inbound traffic (Rr(i,j)) for each customer traffic type (i,j);
• relaying the target rates (Rt(i,j)) for inbound traffic to the limiter means;
• and
• dispatching the admitted inbound traffic (Rp(i,j)) to the servers.