Dynamic capacity management of multiple parallel-connected computing resources

A processing system for processing an incoming request stream, comprising:

• a plurality of computing resources each capable of processing at least a share of the incoming request stream and having a lower-setup-cost state and a higher-setup-cost state;
• a load balancer comprising at least one processor designed and configured to distribute new arrivals within the incoming request stream among said plurality of computing resources;
• a state-change delay timer for each of said plurality of computing resources;
• wherein:
• said state-change delay timer is designed and configured to start running as a function of the corresponding one of said plurality of computing resources becoming idle;
• and the corresponding one of said plurality of computing resources is switched from said lower-setup-cost state to said higher-setup-cost state as a function of a timing-out of said state-change delay timer;
• and a robustness controller designed and configured to determine a minimum number, k.sub.reqd, of said plurality of computing resources to be in said lower-setup-cost state and override ones of said state-change delay timers as needed to keep the minimum number of said plurality of computing resources in said lower-setup-cost state;
• wherein:
• said robustness controller is designed and configured to determine k.sub.reqd as a function of a total number of requests currently distributed among said plurality of computing resources and further as a function of a packing factor and a number, k, of said plurality of computing resources currently in said lower-setup-cost state.

A machine readable storage medium containing machine-executable instructions for controlling a plurality of computing resources in processing an incoming request stream, wherein each of the plurality of computing resources has a lower-setup-cost state and a higher-setup-cost state, said machine-executable instructions comprising:

• a first set of machine-executable instructions for distributing new arrivals in the incoming request stream as a function of a packing factor;
• wherein each of the plurality of computing resources has a state-change delay timer and said machine-executable instructions further includes machine-executable instructions for determining a minimum number, k.sub.reqd, of the plurality of computing resources to be in the lower-setup-cost state and overriding ones of the state-change delay timers as needed to keep the minimum number of the plurality of computing resources in the lower-setup-cost state;
• and wherein:
• said machine-executable instructions further includes machine-executable instructions for determining k.sub.reqd as a function of a total number of requests currently distributed among the plurality of computing resources and further as a function of a packing factor and a number, k, of the plurality of computing resources currently in the lower-setup-cost state.