Load balancer for parallel processors

A hardware logic implemented method for directing incoming processing data units, with such units referred to as packets, to instances of a set of locally hosted application programs referred to as the applications, the method involving:

• at any given one of a set of hardware processing systems, each of which hosts its assigned groups of instances of the applications, producing, for each given one of the applications, (1) an expression of availability of instances hosted at the given processing system that are accepting packets with no destination instance specified, with such packets referred to as NIS packets, and (2) an indication of which instances of the given application are accepting NIS packets;
• and
• by an inter-processing-system load balancer among a set of such load balancers, in case an incoming packet destined for a given one of the applications does not have a destination instance of that given application identified, selecting an appropriate processing system to which to direct the packet, at least in part based on the expressions, from individual processing systems of said set, of availability of instances for the given application for NIS packets;
• and
• at a second-stage load balancer specific to a processing system to which the packet was directed to by the inter-processing-system load balancer, in case an incoming packet destined for a given one of the applications is a NIS packet, determining an appropriate instance of said destination application among the group of instances of that application hosted on the processing system to which to forward the NIS packet, at least in part based on said indication of which instances of said destination application are accepting NIS packets.

A hardware logic system for directing incoming processing data units, with such units referred to as packets, to instances of a locally hosted application program referred to as the application, the system including:

• at any given one of a set of processing systems, each of which hosts its assigned group of instances of the application, a hardware logic subsystem for producing an expression of which instances among said group are accepting packets that have no destination instance specified, with such packets referred to as no-instance-specified (NIS) packets and with such instances accepting NIS packets referred to as available instances;
• and
• at a processing system specific load balancer of a processing system to which the packet was directed to by an inter-processing-system load balancer, a hardware logic subsystem for:
• examining whether the packet has its destination instance identified, and if so, directing the packet toward a processing resource associated with such an identified destination instance, while otherwise, selecting an appropriate instance among the group of instances of the application hosted on the processing system, at least in part based on said expression of which instances among said group are accepting NIS packets, and directing the packet toward a processing resource associated with such selected instance of the application,
• wherein:
• the load balancer has multiple input ports, with an array of multiplexers comprising for each such input port a multiplexer multiplexing packets from a group of buffers storing input packets to the processing system;
• each given multiplexer of the array enters into a trial-on mode of selecting its next one of the buffers to read a new packet from upon reaching the end of a packet it transfers to the load balancer via the input port of the balancer associated with the given multiplexer;
• each of the ports, multiplexers and buffers are identifiable with their unique IDs numbered from 0 upwards within their respective sets;
• and
• in case there are, at a given time:
• H high-fill-available buffers in the group, such that are not being read at the given time, that have one or more packets available for reading and that have their fill levels at or above a monitoring threshold configured for these buffers;
• L low-fill-available buffers in the group, such that are not being read at the given time, that have one or more packets available for reading and that have their fill levels below a monitoring threshold configured for these buffers;
• and K lower ID multiplexers in the trial-on mode than a given multiplexer of the array, wherein K, L and H are non-negative numbers, the buffer from said group for that given multiplexer of said array to read its next packet from is:
• if K is less than H, that high-fill-available buffer, whose count of high-fill-available buffers with lower ID than that buffer equals K;
• and
• else, that high-fill-available or low-fill-available buffer whose count of low-fill-available buffers with lower ID than that buffer equals K minus H.

A hardware logic implemented method for directing incoming processing data units, with such units referred to as packets, to instances of a locally hosted application program referred to as the application, the method involving:

• at any given one of a set of processing systems, each of which hosts its assigned group of instances of the application, producing an expression of which instances among said group are accepting packets with no destination instance specified, with such packets referred to as no-instance-specified (NIS) packets and with such instances accepting NIS packets referred to as available instances;
• and
• at a processing system specific load balancer of a processing system of said set, in case of an incoming NIS packet, selecting an appropriate instance among the group of instances of the application hosted on the processing system to which to direct the packet, at least in part based on said expression of which instances among said group are accepting NIS packets,
• wherein:
• the load balancer has multiple input ports, with an array of multiplexers comprising for each such input port a multiplexer multiplexing packets from a group of buffers storing input packets to the processing system;
• each given multiplexer of the array enters into a trial-on mode of selecting its next one of the buffers to read a new packet from upon reaching the end of a packet it transfers to the load balancer via the input port of the balancer associated with the given multiplexer;
• each of the ports, multiplexers and buffers are identifiable with their unique IDs numbered from 0 upwards within their respective sets;
• and
• in case there are, at a given time:
• H high-fill-available buffers in the group, such that are not being read at the given time, that have one or more packets available for reading and that have their fill levels at or above a monitoring threshold configured for these buffers;
• L low-fill-available buffers in the group, such that are not being read at the given time, that have one or more packets available for reading and that have their fill levels below a monitoring threshold configured for these buffers;
• and K lower ID multiplexers in the trial-on mode than a given multiplexer of the array, wherein K, L and H are non-negative numbers, the buffer from said group for that given multiplexer to read its next packet from is:
• if K is less than H, that high-fill-available buffer, whose count of high-fill-available buffers with lower ID than that buffer equals K;
• and
• else, that high-fill-available or low-fill-available buffer whose count of low-fill-available buffers with lower ID than that buffer equals K minus H.