# US 2024/0406078 A1 · User feedback collection for application qoe prediction

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0406078 A1
- Document ID: US20240406078A1
- Status: Published application
- Kind: A1
- Application number: 18/205,247
- Filed: 2023-06-02
- Published: 2024-12-05
- Earliest claimed: 2023-06-02
- Assignees: Cisco Technology, Inc.
- Applicants: Cisco Technology, Inc.
- Inventors: Pierre-André Savalle; Jean-Philippe VASSEUR; Grégory MERMOUD; Andrea DI PIETRO; Grégoire MAGENDIE
- Web page: https://patentgrep.com/patent/US20240406078A1

## CPC

- H04L 41/5025: by proactively reacting to service quality change, e.g. by reconfiguration after service quality degradation or upgrade
- H04L 41/0816: the condition being an adaptation, e.g. in response to network events
- H04L 41/147: for predicting network behaviour
- H04L 41/5067: Customer-centric QoS measurements
- H04L 43/04: Processing captured monitoring data, e.g. for logfile generation
- H04L 67/75: Indicating network or usage conditions on the user display
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

In one embodiment, a device makes, using a prediction model, a prediction regarding a quality of experience metric for an online application, based on telemetry from a network used to access the network. The device determines a degree of uncertainty associated with the prediction. The device selects one or more users of the online application from which user feedback should be requested, based on the degree of uncertainty. The device requests that the one or more users provide user feedback regarding their satisfaction with the online application.

## Claims

### Claim 1

A method comprising:
making, by a device and using a prediction model, a prediction regarding a quality of experience metric for an online application, based on telemetry from a network used to access the network;
determining, by the device, a degree of uncertainty associated with the prediction;
selecting, by the device, one or more users of the online application from which user feedback should be requested, based on the degree of uncertainty and a query budget that limits how many requests for user feedback can be sent within a given time period; and
requesting, by the device, that the one or more users provide user feedback regarding their satisfaction with the online application.

### Claim 2 (depends on 1)

The method as in claim 1, further comprising:
updating the prediction model using the user feedback from the one or more users.

### Claim 3 (depends on 1)

The method as in claim 1, wherein the device selects the one or more users from among a pool of users onboarded to provide feedback regarding the online application.

### Claim 4 (depends on 3)

The method as in claim 3, further comprising:
onboarding a particular user to provide the feedback in part by requesting that the particular user opt in to providing the feedback.

### Claim 5

(canceled)

### Claim 6 (depends on 1)

The method as in claim 1, wherein the prediction model is used to make a configuration change in the network, based on its predictions.

### Claim 7 (depends on 1)

The method as in claim 1, wherein the device selects the one or more users based in part on their location.

### Claim 8 (depends on 1)

The method as in claim 1, wherein the device determines the degree of uncertainty based in part by simulating user feedback.

### Claim 9 (depends on 1)

The method as in claim 1, wherein the device predicts the quality of experience metric based further in part on telemetry data from the online application.

### Claim 10 (depends on 1)

The method as in claim 1, wherein the device request the user feedback via a chatbot.

### Claim 11

An apparatus, comprising:
one or more network interfaces;
a processor coupled to the one or more network interfaces and configured to execute one or more processes; and
a memory configured to store a process that is executable by the processor, the process when executed configured to:
make, using a prediction model, a prediction regarding a quality of experience metric for an online application, based on telemetry from a network used to access the network;
determine a degree of uncertainty associated with the prediction;
select one or more users of the online application from which user feedback should be requested, based on the degree of uncertainty and a query budget that limits how many requests for user feedback can be sent within a given time period; and
request that the one or more users provide user feedback regarding their satisfaction with the online application.

### Claim 12 (depends on 11)

The apparatus as in claim 11, wherein the process when executed is further configured to:
update the prediction model using the user feedback from the one or more users.

### Claim 13 (depends on 11)

The apparatus as in claim 11, wherein the apparatus selects the one or more users from among a pool of users onboarded to provide feedback regarding the online application.

### Claim 14 (depends on 13)

The apparatus as in claim 13, wherein the process when executed is further configured to:
onboarding a particular user to provide the feedback in part by requesting that the particular user opt in to providing the feedback.

### Claim 15

(canceled)

### Claim 16 (depends on 11)

The apparatus as in claim 11, wherein the prediction model is used to make a configuration change in the network, based on its predictions.

### Claim 17 (depends on 11)

The apparatus as in claim 11, wherein the apparatus selects the one or more users based in part on their location.

### Claim 18 (depends on 11)

The apparatus as in claim 11, wherein the apparatus determines the degree of uncertainty based in part by simulating user feedback.

### Claim 19 (depends on 11)

The apparatus as in claim 11, wherein the apparatus predicts the quality of experience metric based further in part on telemetry data from the online application.

### Claim 20

A tangible, non-transitory, computer-readable medium storing program instructions that cause a device to execute a process comprising:
making, by the device and using a prediction model, a prediction regarding a quality of experience metric for an online application, based on telemetry from a network used to access the network;
determining, by the device, a degree of uncertainty associated with the prediction;
selecting, by the device, one or more users of the online application from which user feedback should be requested, based on the degree of uncertainty and a query budget that limits how many requests for user feedback can be sent within a given time period; and
requesting, by the device, that the one or more users provide user feedback regarding their satisfaction with the online application.
