# US 11,910,249 B2 · Optimizing key allocation during roaming using machine learning

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,910,249 B2
- Document ID: US11910249B2
- Status: Granted
- Kind: B2
- Application number: 17/391,559
- Filed: 2021-08-02
- Published: 2023-02-02
- Granted: 2024-02-20
- Earliest claimed: 2021-08-02
- Assignees: Hewlett Packard Enterprise Development LP
- Applicants: Hewlett Packard Enterprise Development LP
- Inventors: Gopal Gupta; Abhinesh Mishra; Isaac Theogaraj; Sachin Ganu; Bernd Bandemer; Jose Tellado
- Web page: https://patentgrep.com/patent/US11910249B2

## CPC

- H04W 36/0038: {of security context information}
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/092: Reinforcement learning
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06N 20/00: Machine learning
- H04L 41/16: using machine learning or artificial intelligence
- H04L 63/101: {Access control lists [ACL]}
- H04W 12/041: Key generation or derivation
- H04W 12/0431: Key distribution or pre-distribution; Key agreement
- H04W 12/08: Access security
- H04W 36/0061: {of neighbour cell information}
- H04W 88/08: Access point devices
- H04W 36/00835: {Determination of neighbour cell lists}
- H04W 36/38: by fixed network equipment
- H04W 84/12: WLAN [Wireless Local Area Networks]
- H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

Systems and methods are provided for optimizing resource consumption by bringing intelligence to the key allocation process for fast roaming. Specifically, embodiments of the disclosed technology use machine learning to predict which AP a wireless client device will migrate to next. In some embodiments, machine learning may also be used to select a subset of top neighbors from a neighborhood list. Thus, instead of allocating keys for each of the APs on the neighborhood list, key allocation may be limited to the predicted next AP, and the subset of top neighbors. In some embodiments, a reinforcement learning model may be used to dynamically adjust the size of the subset in order to optimize resources while satisfying variable client demand.

## Claims

### Claim 1

A non-transitory computer-readable storage medium including instructions that, when executed by at least one processor of a computing system, cause the computing system to perform a method comprising:
learning migration behavior associated with a client device;
receiving a notification that comprises a current access point and a neighborhood list, wherein the current access point is the access point the client device is currently connected to and the neighborhood list is a list of access points in the neighborhood of the current access point;
predicting an access point the client device will migrate to next based on the current access point and the learned client device migration behavior;
selecting a subset of neighbor access points from the neighborhood list based on the current access point and the learned client device migration behavior, wherein the size of the subset is dynamically adjusted using machine learning;
generating client keys for the predicted next access point and each of the subset of neighbor access points, wherein the client keys are cryptographic keys associated with the client device; and
propagating one of the client keys to each of the predicted next access point and the subset of neighbor access points.

### Claim 2 (depends on 1)

The non-transitory computer-readable storage medium of claim 1, wherein dynamically adjusting the size of the subset using machine learning comprises repeatedly performing the following:
selecting one or more integer values greater than or equal to zero, and less than or equal to the number of access points on the neighborhood list;
evaluating the reward for the selected values; and
choosing the evaluated value which gives the maximum reward.

### Claim 3 (depends on 2)

The non-transitory computer-readable storage medium of claim 2, wherein selecting the one or more integer values greater than or equal to zero, and less than or equal to the number of access points on the neighborhood list, comprises using an epsilon greedy policy.

### Claim 4 (depends on 3)

The non-transitory computer-readable storage medium of claim 3, wherein evaluating the reward for a selected value comprises:
rewarding successful fast roaming attempts; and
punishing failed fast roaming attempts.

### Claim 5 (depends on 4)

The non-transitory computer-readable storage medium of claim 4, wherein:
a successful fast roaming attempt comprises a client device connecting to an access point which has a client key cached; and
a failed fast roaming attempt comprises a client device connecting to an access point which:
the client device has connected to before; and
does not have a client key cached.

### Claim 6 (depends on 1)

The non-transitory computer-readable storage medium of claim 1, wherein the computing system is a cloud-based key management service.

### Claim 7 (depends on 1)

The non-transitory computer-readable storage medium of claim 1, wherein an access point is in the neighborhood of the current access point when the access point's static path loss value relative to the current access point is below a threshold value.

### Claim 8 (depends on 1)

The non-transitory computer-readable storage medium of claim 1, wherein predicting the access point the client device will migrate to next comprises using a Markov Chain Model.

### Claim 9 (depends on 8)

The non-transitory computer-readable storage medium of claim 8, wherein selecting the subset of neighbor access points from the neighborhood list comprises using a Markov Chain Model.

### Claim 10 (depends on 1)

The non-transitory computer-readable storage medium of claim 1, wherein predicting the access point the client device will migrate to next comprises using a deep machine learning based Long Short-Term Memory (LSTM) model.

### Claim 11

A system comprising:
at least one processor; and
a memory storing instructions that, when executed by the at least one processor, cause the system to perform a method comprising:
learning migration behavior associated with a client device;
receiving a notification that comprises a current access point and a neighborhood list, wherein the current access point is the access point the client device is currently connected to and the neighborhood list is a list of access points in the neighborhood of the current access point;
predicting a subset of neighbor access points from the neighborhood list based on the current access point and the learned client device migration behavior, wherein the size of the subset is dynamically adjusted using machine learning; and
generating client keys for the predicted next access point and each of the subset of neighbor access points.

### Claim 12 (depends on 11)

The system of claim 11, wherein dynamically adjusting the size of the subset using machine learning comprises:
selecting one or more integer values greater than zero and less than or equal to the number of access points on the neighborhood list;
evaluating the reward for the selected values;
choosing the evaluated value which gives the maximum reward; and
in response to deterioration of the reward for the chosen value, repeating the selecting, the evaluating, and the choosing, and so on.

### Claim 13 (depends on 12)

The system of claim 12, wherein evaluating the reward for a selected value comprises:
rewarding successful fast roaming iterations; and
punishing failed fast roaming iterations.

### Claim 14 (depends on 13)

The system of claim 13, wherein:
a successful fast roaming iteration comprises one or more successful fast roaming attempts, and zero failed fast roaming attempts;
a failed fast roaming iteration comprises one or more failed fast roaming attempts;
a successful fast roaming attempt comprises a client device connecting to an access point which has a client key cached; and
a failed fast roaming attempt comprises a client device connecting to an access point which:
the client device has connected to before; and
does not have a client key cached.

### Claim 15 (depends on 14)

The system of claim 14, wherein deterioration of the reward for a chosen value comprises at least one of the following:
the number of failed fast roaming iterations increasing relative to the number of successful fast roaming iterations; and
the number of access points at which client keys are cached, which are not connected to during a successful fast roaming iteration, increases.

### Claim 16

A computer-implemented method comprising:
receiving, by a first access point, from a key management service, a cryptographic key associated with a client device, wherein the key management service operates to:
learn migration behavior associated with the client device;
receive a notification that comprises a second access point and a neighborhood list, wherein the second access point is the access point the client device is currently connected to and the neighborhood list is a list of access points in the neighborhood of the second access point;
predict a subset of neighbor access points from the neighborhood list based on the second access point and the learned client device migration behavior, wherein the size of the subset is dynamically adjusted using machine learning and the first access point is included in the subset;
generate cryptographic keys associated with the client device for each of the subset of neighbor access points; and
propagating one of the cryptographic keys associated with the client device to each of subset of neighbor access points; and
caching, by the first access point, the cryptographic key associated with the client device.

### Claim 17 (depends on 16)

The computer-implemented method of claim 16, wherein dynamically adjusting the size of the subset using machine learning comprises:
selecting one or more values between zero and the number of access points on the neighborhood list;
evaluating the reward for the selected values;
choosing the evaluated value which gives the maximum reward; and
in response to deterioration of the reward for the chosen value, repeating the selecting, the evaluating, and the choosing, and so on.

### Claim 18 (depends on 17)

The computer-implemented method of claim 17, wherein evaluating the reward for selected value comprises:
rewarding each successful fast roaming attempt; and
punishing each failed fast roaming attempt.

### Claim 19 (depends on 18)

The computer-implemented method of claim 18, wherein:
a successful fast roaming attempt comprises a client device connecting to an access point which has a cryptographic key associated with a client device cached; and
a failed fast roaming attempt comprises a client device connecting to an access point which:
the client device has connected to before; and
does not have a cryptographic key associated with a client device cached.

### Claim 20 (depends on 19)

The computer implemented-method of claim 19, wherein deterioration of the reward for a chosen value comprises at least one of the following:
the number of failed fast roaming attempts increasing relative to the number of successful fast roaming attempts; and
the number of access points at which keys associated with the client device are cached, which are not connected to during a successful fast roaming attempt, increases.