Software load balancer to maximize utilization

A non-transitory computer-readable medium having stored thereon instructions executable by a computer system to cause the computer system to perform operations comprising:

• assigning a first sub-group of a plurality of requests for computing service to a first computing node in a plurality of computing nodes, wherein a second computing node in the plurality of computing nodes is idling when the first sub-group of the plurality of requests is assigned to the first computing node based on first feedback received from the first computing node indicating that the first computing node has not reached a work assignment termination condition;
• subsequent to the assigning the first sub-group of the plurality of requests for computing services to the first computing node, obtaining, from the first computing node, second feedback related to a capacity of the first computing node to process requests according to a set of predetermined servicing criteria;
• determining that the first computing node has reached the work assignment termination condition based on the second feedback, wherein the work assignment termination condition indicates at least one of (i) an inability to process additional requests for computing service according to a service level agreement (SLA), or (ii) an inability to accept the additional requests for computing service;
• in response to determining that the first computing node has reached the work assignment termination condition, activating the second computing node;
• and
• assigning a second sub-group of the plurality of requests to the second computing node.

A method, comprising:

• assigning a first sub-group of a plurality of requests for computing service to a first computing node in a plurality of computing nodes, wherein a second computing node in the plurality of computing nodes is inactive when the first sub-group of the plurality of requests is assigned to the first computing node based on first feedback received from the first computing node indicating that the first computing node has not reached a work assignment termination condition;
• obtaining, from the first computing node, second feedback related to a capacity of the first computing node to process requests according to a set of predetermined servicing criteria;
• determining that the first computing node has reached the work assignment termination condition based on the second feedback;
• in response to determining that the first computing node has reached the work assignment termination condition, activating the second computing node;
• and
• assigning a second sub-group of the plurality of requests to the second computing node.