# US 2022/0046083 A1 · Performing load balancing self adjustment within an application environment

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0046083 A1
- Document ID: US20220046083A1
- Status: Published application
- Kind: A1
- Application number: 17/393,960
- Filed: 2021-08-04
- Published: 2022-02-10
- Earliest claimed: 2020-08-05
- Assignees: Avesha, Inc.
- Applicants: Avesha, Inc.
- Inventors: Raj Nair; Prabhudev Navali; Sudhir Halbhavi; Chin-Cheng Wu
- Web page: https://patentgrep.com/patent/US20220046083A1

## CPC

- H04L 67/1001: for accessing one among a plurality of replicated servers
- H04L 67/1008: based on parameters of servers, e.g. available memory or workload (monitoring of computer activity G06F11/30)
- G06F 9/5077: {Logical partitioning of resources; Management or configuration of virtualized resources (specific details on emulation or internal functioning of virtual machines G06F9/455)}
- G06N 3/092: Reinforcement learning
- G06N 20/00: Machine learning
- H04L 41/0816: the condition being an adaptation, e.g. in response to network events
- H04L 47/20: Traffic policing
- H04L 67/02: based on web technology, e.g. hypertext transfer protocol [HTTP]
- H04L 67/101: based on network conditions
- H04L 67/1023: based on a hash applied to IP addresses or costs
- H04L 67/1029: using data related to the state of servers by a load balancer
- H04L 67/34: {involving the movement of software or configuration parameters (network booting or remote initial program loading [RIPL] G06F9/4416)}
- H04L 67/60: Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources (admission control or resource allocation H04L47/70)
- H04N 21/2187: Live feed
- H04N 21/42203: {sound input device, e.g. microphone}
- H04N 21/4312: {involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations}
- H04N 21/4334: {Recording operations (recording of a television signal H04N5/76; arrangements for recording or accumulating broadcast information or broadcast-related information H04H60/27)}
- H04N 21/466: Learning process for intelligent management, e.g. learning user preferences for recommending movies {(services using the results of monitoring in broadcast systems H04H60/61)}
- H04N 21/47217: {for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks (specific graphical features in visual interfaces H04N21/4312)}
- G06N 3/08: Learning methods
- G06N 20/20: Ensemble learning
- G08B 13/19671: {Addition of non-video data, i.e. metadata, to video stream}
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS

## Abstract

A technique performs load balancing self-adjustment within an application environment. The technique involves, while nodes of the application environment load balance traffic among clusters that provide services for an application in accordance with a first load balancing configuration, sensing application environment metrics. The technique further involves performing a self-adjustment operation that generates a second load balancing configuration based on the application environment metrics, the second load balancing configuration being different from the first load balancing configuration. The technique further involves deploying the second load balancing configuration among the nodes to enable the nodes to load balance the traffic among the clusters that provide the services for the application in accordance with second load balancing configuration in place of the first load balancing configuration.

## Claims

### Claim 1

A method of performing load balancing self-adjustment within an application environment, the method comprising:
while nodes of the application environment load balance traffic among clusters that provide services for an application in accordance with a first load balancing configuration, sensing application environment metrics;
performing a self-adjustment operation that generates a second load balancing configuration based on the application environment metrics, the second load balancing configuration being different from the first load balancing configuration; and
deploying the second load balancing configuration among the nodes to enable the nodes to load balance the traffic among the clusters that provide the services for the application in accordance with second load balancing configuration in place of the first load balancing configuration.

### Claim 2 (depends on 1)

The method of claim 1 wherein the first load balancing configuration includes a first set of load balancing weights; and
wherein sensing the application environment metrics includes:
obtaining the application environment metrics while the nodes of the application environment load balance the traffic among the clusters that provide the services in accordance with the first set of load balancing weights.

### Claim 3 (depends on 2)

The method of claim 2 wherein the second load balancing configuration includes a second set of load balancing weights, at least some of the load balancing weights of the second set being different from respective load balancing weights of the first set; and
wherein deploying the second load balancing configuration includes:
re-configuring the nodes of the application environment to load balance the traffic among the clusters that provide the services in accordance with the second set of load balancing weights in place of the first set of load balancing weights.

### Claim 4 (depends on 3)

The method of claim 3 wherein the clusters include a first microservice cluster and a second microservice cluster, each of the first microservice cluster and the second microservice cluster providing a same microservice for the application;
wherein a particular node initially load balances microservice requests among the first microservice cluster and the second microservice cluster in accordance with the first set of load balancing weights while the application environment metrics are sensed; and
wherein re-configuring the nodes includes:
changing operation of the particular node to load balance microservice requests among the first microservice cluster and the second microservice cluster in accordance with the second set of load balancing weights.

### Claim 5 (depends on 3)

The method of claim 3 wherein the clusters include a plurality of microservice clusters which forms a service mesh, the plurality of microservice clusters including groups of clusters providing the same microservice.

### Claim 6 (depends on 3)

The method of claim 3 wherein the clusters include a first service cluster and a second service cluster, each of the first service cluster and the second service cluster performing a same application routine for the application;
wherein a particular node initially load balances application routine requests among the first service cluster and the second service cluster in accordance with the first set of load balancing weights while the application environment metrics are sensed; and
wherein re-configuring the nodes includes:
changing operation of the particular node to load balance application routine requests among the first service cluster and the second service cluster in accordance with the second set of load balancing weights.

### Claim 7 (depends on 1)

The method of claim 1 wherein the application environment metrics includes application environment state information; and
wherein performing the self-adjustment operation includes:
entering the application environment state information into a policy engine constructed and arranged to generate load balancing configurations, the policy engine generating the second load balancing configuration based on the entered application environment state information.

### Claim 8 (depends on 7)

The method of claim 7 wherein performing the self-adjustment operation further includes:
starting a sample timer that is configured to expire at a predefined sample time, the application environment state information being received into storage from the nodes after starting the sample timer and prior to expiration of sample timer at the predefined sample time, and
wherein the application environment state information is entered from the storage into the policy engine upon expiration of the sample timer at the predefined sample time.

### Claim 9 (depends on 8)

The method of claim 8 wherein the policy engine includes an algorithmic policy model;
wherein the sample of the application environment state information includes:
respective network latency samples, throughput samples, and application resource utilization samples from the nodes of the application environment; and
wherein entering the sample of the application environment state information into the policy engine includes:
applying the respective network latency samples, throughput samples, and application resource utilization samples to the algorithmic policy model to create the second load balancing configuration.

### Claim 10 (depends on 8)

The method of claim 8, further comprising:
continuing to sense the application environment metrics to form a series of application environment state samples;
continuing to perform the self-adjustment operation periodically in response to operation of the sample timer to form a series of new load balancing configurations based on the series of application environment state samples; and
deploying the series of new load balancing configurations among the nodes to enable the nodes to load balance the traffic base on the series of new load balancing configurations.

### Claim 11 (depends on 10)

The method of claim 10, further comprising:
computing a series of rewards based on the series of application environment state samples, the series of rewards identifying a series of application environment behavior changes over time.

### Claim 12 (depends on 11)

The method of claim 11 wherein the series of new load balancing configurations includes a series of actions defining a series of load balancing adjustments made to the nodes of the application environment over time; and
wherein the method further comprises:
forming a series of state-action-reward entries based on (i) the series of application environment state samples, (ii) the series of actions, and (iii) the series of rewards, and
storing the series of state-action-reward entries in a state-action-reward repository.

### Claim 13 (depends on 12)

The method of claim 12, further comprising:
starting an entry counter that is configured to count to a predefined entry count, and
in response to the entry counter reaching the predefined entry count, performing an update operation that updates the policy engine based on the series of state-action-reward entries stored in the state-action-reward repository.

### Claim 14 (depends on 13)

The method of claim 13 wherein the policy engine includes a deep learning model configured to output a set of actions to be taken for a given input state; and
wherein performing the update operation includes:
training the deep learning model using the series of state-action-reward entries stored in the state-action-reward repository.

### Claim 15 (depends on 1)

The method of claim 1 wherein the nodes of the application environment include enforcement points that form a microservice mesh; and
wherein deploying the second load balancing configuration includes:
programming the enforcement points with respective load balancing policies that direct the enforcement points to load balance microservice requests in accordance with the respective load balancing policies.

### Claim 16 (depends on 15)

The method of claim 15 wherein programming the enforcement points with the respective load balancing policies includes:
configuring a set of enforcement points to issue ingested microservice requests in accordance with round robin based load balancing.

### Claim 17 (depends on 15)

The method of claim 15 wherein programming the enforcement points with the respective load balancing policies includes:
configuring a set of enforcement points to issue ingested microservice requests in accordance with reinforcement learning based load balancing.

### Claim 18

Electronic circuitry, comprising:
memory; and
control circuitry coupled to the memory, the memory storing instructions which, when carried out by the control circuitry, cause the control circuitry to perform a method of:
while nodes of an application environment load balance traffic among clusters that provide services for an application in accordance with a first load balancing configuration, sensing application environment metrics,
performing a self-adjustment operation that generates a second load balancing configuration based on the application environment metrics, the second load balancing configuration being different from the first load balancing configuration, and
deploying the second load balancing configuration among the nodes to enable the nodes to load balance the traffic among the clusters that provide the services for the application in accordance with second load balancing configuration in place of the first load balancing configuration.

### Claim 19

A computer program product having a non-transitory computer readable medium which stores a set of instructions to perform load balancing self-adjustment within an application environment; the set of instructions, when carried out by computerized circuitry, causing the computerized circuitry to perform a method of:
while nodes of the application environment load balance traffic among clusters that provide services for an application in accordance with a first load balancing configuration, sensing application environment metrics;
performing a self-adjustment operation that generates a second load balancing configuration based on the application environment metrics, the second load balancing configuration being different from the first load balancing configuration; and
deploying the second load balancing configuration among the nodes to enable the nodes to load balance the traffic among the clusters that provide the services for the application in accordance with second load balancing configuration in place of the first load balancing configuration.
