Dynamic load balancing in reactive systems

A method for load balancing in a computer system, the method comprising:

• providing a plurality of services, the plurality of services comprising at least one consumer service and an event stream service for supplying a stream of events from a producer service to a consumer service, each consumer service having an event capacity;
• and
• providing a service mesh, the service mesh comprising a data plane and a control plane, wherein the control plane is configured to create and remove instances of the plurality of services, including creating a proxy instance for each service instance, and wherein the data plane is configured to define interrelations between the proxy instances and hence indirectly between the service instances,
• wherein the control plane is operable to manage provisioning of consumer service instances according to computer system needs by:
• defining a specified event capacity to be provided by the at least one consumer services instances collectively;
• estimating a current event capacity of the computer system by measuring and summing the event capacities of each consumer service instance;
• determining any shortfalls of the current event capacity from the specified burst capacity;
• and
• adding at least one further consumer service instance conditional on determining a shortfall in the event capacity sufficient to eliminate the shortfall;
• wherein the event capacity comprises an event burst capacity and an event throughput capacity, the specified and current event capacities thus each having burst and throughput elements, and wherein any shortfall also has burst and throughput elements;
• wherein said conditional adding of at least one further consumer service instance is conditional on determining a respective shortfall in at least one of the event burst capacity and the event throughput capacity sufficient to eliminate the respective shortfall or each shortfall element;
• wherein said measuring of the event capacities of each consumer service instance is based on windowing and batching information extracted by the control plane from communications to the proxy instance of that consumer service instance;
• and
• wherein both the windowing information and the batching information are used to measure the event burst capacity, whereas the batching information but not the windowing information is used to measure the event throughput capacity.

A computer program product storing a computer program comprising software code portions, which, when said program is run on a computing device, cause the computing device to perform a method for load balancing in a computer system, the method comprising:

• providing a plurality of services, the plurality of services comprising at least one consumer service and an event stream service for supplying a stream of events from a producer service to a consumer service, each consumer service having an event capacity;
• and
• providing a service mesh, the service mesh comprising a data plane and a control plane, wherein the control plane is configured to create and remove instances of the services, including creating a proxy instance for each service instance, and wherein the data plane is configured to define interrelations between the proxy instances and hence indirectly between the service instances,
• operating the control plane to manage provisioning of consumer service instances according to computer system needs by:
• defining a specified event capacity to be provided by the at least one consumer services collectively;
• estimating a current event capacity of the computer system by measuring and summing the event capacities of each consumer service instance;
• determining any shortfall of the current event capacity from the specified burst capacity;
• and
• adding at least one further consumer service instance conditional on determining a shortfall in the event capacity sufficient to eliminate the shortfall;
• wherein the event capacity comprises an event burst capacity and an event throughput capacity, the specified and current event capacities thus each having burst and throughput elements, and wherein any shortfall also has burst and throughput elements;
• wherein said conditional adding of at least one further consumer service instance is conditional on determining a respective shortfall in at least one of the event burst capacity and the event throughput capacity sufficient to eliminate the respective shortfall or each shortfall element;
• wherein said measuring of the event capacities of each consumer service instance is based on windowing and batching information extracted by the control plane from communications to the proxy instance of that consumer service instance;
• and
• wherein both the windowing information and the batching information are used to measure the event burst capacity, whereas the batching information but not the windowing information is used to measure the event throughput capacity.