# US 2025/0156750 A1 · Reinforcement learning for machine learning models using dynamic confidence thresholds

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0156750 A1
- Document ID: US20250156750A1
- Status: Published application
- Kind: A1
- Application number: 18/508,680
- Filed: 2023-11-14
- Published: 2025-05-15
- Earliest claimed: 2023-11-14
- Assignees: Optum, Inc.
- Applicants: Optum, Inc.
- Inventors: Ayan Sengupta; Sudhanshu Sharma; Julie Zhu; Elizabeth Patterson Cohn
- Web page: https://patentgrep.com/patent/US20250156750A1

## CPC

- G06N 20/00: Machine learning
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Various embodiments of the present disclosure provide reinforcement learning for machine learning using dynamic confidence thresholds. In one example, an embodiment provides for generating a plurality of training datasets for a machine learning model by augmenting a labeled dataset for the machine learning model with a synthetic labeled dataset, generating a plurality of retrained model versions of the machine learning model based on the plurality of training datasets, generating a reward indicator for a retrained model version of the plurality of retrained versions of the machine learning model based on a comparison between a validation dataset for the machine learning model and a respective output dataset for the retrained model version, and modifying the defined confidence threshold based on the reward indicator for the retrained model version to generate a modified confidence threshold for the machine learning model.

## Claims

### Claim 1

A computer-implemented method, the computer-implemented method comprising:
generating, by one or more processors, a plurality of training datasets for a machine learning model by augmenting a labeled dataset for the machine learning model with a synthetic labeled dataset that (i) is related to unlabeled data and (ii) comprises one or more data inferences by the machine learning model that satisfy a defined confidence threshold;
generating, by the one or more processors, a plurality of retrained model versions of the machine learning model based on the plurality of training datasets;
generating, by the one or more processors, a reward indicator for a retrained model version of the plurality of retrained versions of the machine learning model based on a comparison between a validation dataset for the machine learning model and a respective output dataset for the retrained model version;
modifying, by the one or more processors and using a multi-armed bandit model, the defined confidence threshold based on the reward indicator for the retrained model version to generate a modified confidence threshold for the machine learning model; and
initiating, by the one or more processors, the performance of the machine learning model based on the modified confidence threshold.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, further comprising:
generating the machine learning model by training the machine learning model based on (i) the labeled dataset and (ii) the validation dataset.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, further comprising:
generating the synthetic labeled dataset by applying the machine learning model to the unlabeled data.

### Claim 4 (depends on 1)

The computer-implemented method of claim 1, further comprising:
generating the defined confidence threshold based on a predefined performance metric.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, further comprising:
initiating the performance of the multi-armed bandit model based on a confidence threshold set that comprises a plurality of candidate confidence parameters for the machine learning model.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, further comprising:
initiating the performance of the multi-armed bandit model based on an upper confidence bound (UCB) prediction with respect to the respective reward indicators.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, wherein initiating the performance of the machine learning model comprises:
modifying one or more hyperparameter configurations of the machine learning model based on the modified confidence threshold.

### Claim 8 (depends on 1)

The computer-implemented method of claim 1, wherein initiating the performance of the machine learning model comprises:
initiating the performance of one or more prediction-based actions via the machine learning model and the modified confidence threshold.

### Claim 9 (depends on 1)

The computer-implemented method of claim 1, wherein initiating the performance of the machine learning model comprises:
generating one or more labels for a training dataset via the machine learning model and the modified confidence threshold.

### Claim 10

A computing system comprising memory and one or more processors communicatively coupled to the memory, the one or more processors configured to:
generate a plurality of training datasets for a machine learning model by augmenting a labeled dataset for the machine learning model with a synthetic labeled dataset that (i) is related to unlabeled data and (ii) comprises one or more data inferences by the machine learning model that satisfy a defined confidence threshold;
generate a plurality of retrained model versions of the machine learning model based on the plurality of training datasets;
generate a reward indicator for a retrained model version of the plurality of retrained versions of the machine learning model based on a comparison between a validation dataset for the machine learning model and a respective output dataset for the retrained model version;
modify, using a multi-armed bandit model, the defined confidence threshold based on the reward indicator for the retrained model version to generate a modified confidence threshold for the machine learning model; and
initiate the performance of the machine learning model based on the modified confidence threshold.

### Claim 11 (depends on 10)

The computing system of claim 10, wherein the one or more processors are further configured to:
generate the machine learning model by training the machine learning model based on (i) the labeled dataset and (ii) the validation dataset.

### Claim 12 (depends on 10)

The computing system of claim 10, wherein the one or more processors are further configured to:
generate the synthetic labeled dataset by applying the machine learning model to the unlabeled data.

### Claim 13 (depends on 10)

The computing system of claim 10, wherein the one or more processors are further configured to:
generate the defined confidence threshold based on a predefined performance metric.

### Claim 14 (depends on 10)

The computing system of claim 10, wherein the one or more processors are further configured to:
initiate the performance of the multi-armed bandit model based on a confidence threshold set that comprises a plurality of candidate confidence parameters for the machine learning model.

### Claim 15 (depends on 10)

The computing system of claim 10, wherein the one or more processors are further configured to:
initiate the performance of the multi-armed bandit model based on an upper confidence bound (UCB) prediction with respect to the respective reward indicators.

### Claim 16

One or more non-transitory computer-readable storage media including instructions that, when executed by one or more processors, cause the one or more processors to:
generate a plurality of training datasets for a machine learning model by augmenting a labeled dataset for the machine learning model with a synthetic labeled dataset that (i) is related to unlabeled data and (ii) comprises one or more data inferences by the machine learning model that satisfy a defined confidence threshold;
generate a plurality of retrained model versions of the machine learning model based on the plurality of training datasets;
generate a reward indicator for a retrained model version of the plurality of retrained versions of the machine learning model based on a comparison between a validation dataset for the machine learning model and a respective output dataset for the retrained model version;
modify, using a multi-armed bandit model, the defined confidence threshold based on the reward indicator for the retrained model version to generate a modified confidence threshold for the machine learning model; and
initiate the performance of the machine learning model based on the modified confidence threshold.

### Claim 17 (depends on 16)

The one or more non-transitory computer-readable storage media of claim 16, wherein the instructions further cause the one or more processors to:
generate the synthetic labeled dataset by applying the machine learning model to the unlabeled data.

### Claim 18 (depends on 16)

The one or more non-transitory computer-readable storage media of claim 16, wherein the instructions further cause the one or more processors to:
generate the defined confidence threshold based on a predefined performance metric.

### Claim 19 (depends on 16)

The one or more non-transitory computer-readable storage media of claim 16, wherein the instructions further cause the one or more processors to:
initiate the performance of the multi-armed bandit model based on a confidence threshold set that comprises a plurality of candidate confidence parameters for the machine learning model.

### Claim 20 (depends on 16)

The one or more non-transitory computer-readable storage media of claim 16, wherein the instructions further cause the one or more processors to:
initiate the performance of the multi-armed bandit model based on an upper confidence bound (UCB) prediction with respect to the respective reward indicators.
