# US 12,603,837 B1 · Method and system for optimization of network traffic based on reinforcement learning

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,603,837 B1
- Document ID: US12603837B1
- Status: Granted
- Kind: B1
- Application number: 19/241,182
- Filed: 2025-06-17
- Granted: 2026-04-14
- Earliest claimed: 2025-06-17
- Assignees: CyKor LLC
- Applicants: CyKor LLC
- Inventors: Mark Mitchiner
- Web page: https://patentgrep.com/patent/US12603837B1

## CPC

- H04L 45/38: {Flow based routing}
- H04L 41/0894: Policy-based network configuration management
- H04L 41/16: using machine learning or artificial intelligence
- H04L 43/0829: Packet loss
- H04L 43/087: Jitter
- H04L 43/0876: Network utilisation, e.g. volume of load or congestion level
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

A system for optimizing network traffic distribution including a control plane connected to a data plane. The control plane includes one or more processors and memory storing instructions that, when executed by the one or more processors, cause the control plane to: implement a reinforcement learning module configured to: model a network state space comprising interface characteristics and traffic characteristics; define an action space for allocating specific flows to specific interfaces; establish a reward function that encourages efficient utilization of bandwidth while penalizing packet loss, latency, and jitter; learn optimal flow-to-interface allocations based on the modeled state space, action space, and reward function; and implement a flow allocation module configured to generate flow allocation policies based on the learned optimal flow-to-interface allocations. The data plane including one or more network devices configured to route network traffic flows based on the implemented flow allocation policies.

## Claims

### Claim 1

A system for optimizing network traffic distribution across an equal-cost multi-path (ECMP) set of interfaces, comprising:
a control plane associated with a set of ECMP interfaces comprising:
one or more processors; and
memory storing instructions that, when executed by the one or more processors, cause the control plane to:
perform reinforcement learning operations to:
model a network state space comprising interface characteristics including bandwidth capacity per interface representing bins having capacity constraints, and traffic characteristics including size of each traffic flow representing items of varying sizes to be packed into the bins;
define an action space for allocating specific flows to specific interfaces, wherein each action represents a decision to assign a specific flow to a specific interface to pack the flow into the interface while respecting the capacity constraints;
establish a reward function that encourages efficient utilization of bandwidth while penalizing packet loss, latency, and jitter, wherein the reward function comprises a utilization reward based on a ratio of used bandwidth to total available bandwidth and a penalty that is a weighted combination of packet loss rate, latency, and jitter; and
learn optimal flow-to-interface allocations based on the modeled state space, action space, and reward function; and
perform flow allocation operations to:
generate flow allocation policies based on the learned optimal flow-to-interface allocations; and
dynamically update the flow allocation policies in response to changes in network conditions detected based on collected network statistics, wherein the updated flow allocation policies reassign one or more flows from a first interface to a second interface from the set of ECMP interfaces based on the reward function, the action space, and the state space; and
a data plane associated with the set of ECMP interfaces comprising:
one or more network devices configured to:
implement the flow allocation policies generated by the flow allocation operations;
collect the network statistics including bandwidth utilization, packet loss rates, latency, and jitter for each interface;
transmit the collected network statistics to the control plane in real time; and
route network traffic flows based on the implemented flow allocation policies.

### Claim 2 (depends on 1)

The system of claim 1, wherein the interface characteristics modeled by the reinforcement learning operations comprise bandwidth utilization per interface, current packet loss rates per interface, and current latency and jitter per interface.

### Claim 3 (depends on 1)

The system of claim 1, wherein the traffic characteristics modeled by the reinforcement learning operations comprise size and type of each traffic flow, and priority of traffic flows.

### Claim 4 (depends on 1)

The system of claim 1, wherein the reward function established by the reinforcement learning operations is defined as a combination of a positive reward for effectively utilizing available bandwidth and a penalty for packet loss, excessive latency, and jitter.

### Claim 5 (depends on 1)

The system of claim 1, wherein the reinforcement learning operations implement Q-Learning or a variant thereof.

### Claim 6 (depends on 5)

The system of claim 5, wherein the reinforcement learning operations use a deep reinforcement learning framework with a neural network to approximate Q-values or policy.

### Claim 7 (depends on 6)

The system of claim 6, wherein the neural network of the reinforcement learning operations takes as input the interface characteristics, traffic characteristics, and current flow-to-interface allocations, and outputs a probability distribution over possible flow-to-interface allocations.

### Claim 8 (depends on 6)

The system of claim 6, wherein the reinforcement learning operations use experience replay to store transitions in a replay buffer and sample the stored transitions for training.

### Claim 9 (depends on 6)

The system of claim 6, wherein the reinforcement learning operations use a separate target network to stabilize training by periodically updating its weights.

### Claim 10 (depends on 6)

The system of claim 6, wherein the reinforcement learning operations minimize a temporal difference error as a loss function for training the neural network.

### Claim 11 (depends on 1)

The system of claim 1, wherein the control plane further comprises simulation operations configured to simulate network traffic flows with varying characteristics and simulate stochastic changes in interface conditions.

### Claim 12 (depends on 1)

The system of claim 1, wherein the reinforcement learning operations use an ε-greedy policy for exploration or a stochastic policy for proximal policy optimization.

### Claim 13 (depends on 8)

The system of claim 8, wherein the reinforcement learning operations use prioritized experience replay to prioritize transitions with higher temporal difference error for sampling.

### Claim 14 (depends on 1)

The system of claim 1, wherein the reinforcement learning operations use a multi-agent approach where each agent manages flows for a specific interface, and agents coordinate to optimize global performance.

### Claim 15 (depends on 1)

The system of claim 1, wherein the control plane further comprises predictive operations configured to forecast traffic patterns and proactively adjust allocations.

### Claim 16 (depends on 1)

The system of claim 1, wherein the reinforcement learning operations are implemented using a deep Q-network (DQN) architecture.

### Claim 17 (depends on 1)

The system of claim 1, wherein the reinforcement learning operations periodically retrain to adapt to long-term changes in network conditions.

### Claim 18 (depends on 1)

The system of claim 1, wherein the flow allocation operations operate in the control plane to make flow allocation decisions, the one or more network devices are connected by links to remote network devices at the interfaces of the one or more network devices, and the links connected to the remote network devices are modeled as an equal cost multi-path system.

### Claim 19

A method for optimizing network traffic distribution across an equal-cost multi-path (ECMP) set of interfaces, comprising:
in a control plane, optimizing network traffic distribution across an equal-cost multi-path (ECMP) set of interfaces by:
implementing reinforcement learning (RL) operations, including:
modeling a network state space comprising interface characteristics including bandwidth capacity per interface representing bins having capacity constraints, and traffic characteristics including size of each traffic flow representing items of varying sizes to be packed into the bins;
defining an action space for allocating specific flows to specific interfaces, wherein each action represents a decision to assign a specific flow to a specific interface to pack the flow into the interface while respecting the capacity constraints;
establishinga reward function that encourages efficient utilization of bandwidth while penalizing packet loss, latency, and jitter, wherein the reward function comprises a utilization reward based on a ratio of used bandwidth to total available bandwidth and a penalty that is a weighted combination of packet loss rate, latency, and litter;
learning optimal flow-to-interface allocations based on the modeled state space, action space, and reward function; and
implementing flow allocation operations, including:
generating flow allocation policies based on the learned optimal flow-to-interface allocations; and
dynamically updating the flow allocation policies in response to changes in network conditions detected based on collected network statistics, wherein the updated flow allocation policies reassign one or more flows from a first interface to a second interface based on available bandwidth; and
implementing, in a data pane, the flow allocation policies generated by the flow allocation operations;
collecting, by at least one network device of the data plane, network statistics including bandwidth utilization, packet loss rates, latency, and jitter for each interface;
transmitting the collected network statistics to the control plane in real time; and
routing network traffic flows based on the implemented flow allocation policies.

### Claim 20

A non-transitory computer-readable medium comprising instructions, that when read by a processor, cause the processor to perform:
in a control plane, optimizing network traffic distribution across an equal-cost multi-path (ECMP) set of interfaces by:
implementing reinforcement learning (RL); operations, including:
modeling a network state space comprising interface characteristics including bandwidth capacity per interface representing bins having capacity constraints, and and traffic characteristics including size of each traffic flow representing items of varying sizes to be packed into the bins;
defining an action space for allocating specific flows to specific interfaces, wherein each action represents a decision to assign a specific flow to a specific interface to pack the flow into the interface while respecting the capacity constraints;
establishing a reward function that encourages efficient utilization of bandwidth while penalizing packet loss, latency, and jitter, wherein the reward function comprises a utilization reward based on a ratio of used bandwidth to total available bandwidth and a Penalty that is a weighted combination of packet loss rate, latency, and litter;
learning optimal flow-to-interface allocations based on the modeled state space, action space, and reward function; and
implementing flow allocation operations, including:
generating flow allocation policies based on the learned optimal flow-to-interface allocations;
dynamically updating the flow allocation policies in response to changes in network conditions detected based on collected network statistics, wherein the updated flow allocation policies reassign one or more flows from a first interface to a second interface based on available bandwidth; and
implementing, in a data pane, the flow allocation policies generated by the flow allocation operations;
collecting, by at least one network device of the data plane, network statistics including bandwidth utilization, packet loss rates, latency, and jitter for each interface;
transmitting the collected network statistics to the control plane in real time; and
routing network traffic flows based on the implemented flow allocation policies.