# US 2026/0073231 A1 · Safe meta-reinforcement learning (safe meta-rl) prompting for machine learning

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0073231 A1
- Document ID: US20260073231A1
- Status: Published application
- Kind: A1
- Application number: 18/882,709
- Filed: 2024-09-11
- Published: 2026-03-12
- Earliest claimed: 2024-09-11
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Alexander Zadorojniy; Ashlesha Akella; Dharmashankar Subramanian
- Web page: https://patentgrep.com/patent/US20260073231A1

## CPC

- G06N 3/092: Reinforcement learning
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06N 20/00: Machine learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

One or more parameters of a meta-reinforcement learning (meta-RL) model are updated, wherein the updated parameters comprise an update to an action space. A prompt is generated using the reinforcement learning model with the updated parameters for a large language model. The prompt is provided to the large language model to perform a downstream task.

## Claims

### Claim 1

A computer-implemented method comprising:
updating one or more parameters of a meta-reinforcement learning (meta-RL) model, wherein the updated parameters comprise an update to an action space;
generating a prompt, using the reinforcement learning model with the updated parameters, for a large language model; and
providing the prompt to the large language model to perform a downstream task.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein the updating of the one or more parameters includes reducing a size of the action space of the reinforcement learning model.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein the updating of the one or more parameters includes increasing a prompt length parameter of the reinforcement learning model.

### Claim 4 (depends on 3)

The computer-implemented method of claim 3, wherein the increasing of the length parameter is performed in conjunction with a reducing of a size of the action space for the reinforcement learning model.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein the use of the reinforcement learning model is based on an immediate risk risk(s′, a) defined as:
risk
(
s
′
,
a
)
=
max
⁢
{
A
tg
-
A
c
⁢
r
,
0.001
}
/
(
T
tr
-
t
)
where t<Ttr−1, Atg is an accuracy target, Acr is a current batch iteration accuracy, Ttr is a single trial training time or training iterations, srem=Ttr−t is a new state variable, S′=S∪srem is a new state space and wherein the immediate risk risk(s′, a) is defined over a batch of rows of a meta data table.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, wherein the generating of the prompt using the reinforcement learning model comprises inputting a task and meta data table, wherein the task identifies the downstream task.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, wherein the updating of the one or more parameters of the reinforcement learning model is halted after a specified number of iterations, after a specified number of seconds or after a specified size reduction of the reduced dictionary.

### Claim 8 (depends on 1)

The computer-implemented method of claim 1, further comprising providing the generated prompt and an instance as input to the large language model.

### Claim 9 (depends on 1)

The computer-implemented method of claim 1, further comprising evaluating tokens of the generated prompt to identify tokens having a higher probability of success.

### Claim 10 (depends on 1)

The computer-implemented method of claim 1, further comprising:
analyzing a final output of the large language model using a reward function tailored for data wrangling tasks and a risk function tailored for the data wrangling tasks; and
providing, to the reinforcement learning model, a result of applying the reward function as part of a reinforcement learning process.

### Claim 11 (depends on 10)

The computer-implemented method of claim 10, wherein a risk-aware objective function is defined as:
J
⁡
(
θ
)
=
E
M
i
[
τ
⁡
(
π
)
∂
(
π
)
ω
⁢
❘
"\[LeftBracketingBar]"
f
θ
,
M
i
]
,
where EMi is an expectation over Markov decision processes M, θ is a meta-parameter, fθ is a mapping to inner loop parameters, τ(π) is an inner loop expectation of reward over policy π, ∂(π)ω is an expectation of risk over policy π.

### Claim 12

A computer program product, comprising:
one or more tangible computer-readable storage media and program instructions stored on at least one of the one or more tangible computer-readable storage media, the program instructions executable by a processor, the program instructions comprising:
updating one or more parameters of a meta-reinforcement learning (meta-RL) model, wherein the updated parameters comprise an update to an action space;
generating a prompt, using the reinforcement learning model with the updated parameters, for a large language model; and
providing the prompt to the large language model to perform a downstream task.

### Claim 13 (depends on 12)

The computer program product of claim 12, wherein the updating of the one or more parameters includes reducing a size of the action space of the reinforcement learning model.

### Claim 14

A system comprising:
a memory; and
at least one processor, coupled to said memory, and operative to perform operations comprising:
updating one or more parameters of a meta-reinforcement learning (meta-RL) model, wherein the updated parameters comprise an update to an action space;
generating a prompt, using the reinforcement learning model with the updated parameters, for a large language model; and
providing the prompt to the large language model to perform a downstream task.

### Claim 15 (depends on 14)

The system of claim 14, wherein the updating of the one or more parameters includes reducing a size of the action space of the reinforcement learning model.

### Claim 16 (depends on 14)

The system of claim 14, wherein the updating of the one or more parameters includes increasing a prompt length parameter of the reinforcement learning model.

### Claim 17 (depends on 16)

The system of claim 16, wherein the increasing of the length parameter is performed in conjunction with a reducing of a size of the action space for the reinforcement learning model.

### Claim 18 (depends on 14)

The system of claim 14, the operations further comprising providing the generated prompt and an instance as input to the large language model.

### Claim 19 (depends on 14)

The system of claim 14, the operations further comprising evaluating tokens of the generated prompt to identify tokens having a higher probability of success.

### Claim 20 (depends on 14)

The system of claim 14, the operations further comprising:
analyzing a final output of the large language model using a reward function tailored for data wrangling tasks and a risk function tailored for the data wrangling tasks; and
providing, to the reinforcement learning model, a result of applying the reward function as part of a reinforcement learning process.
