# US 2026/0086567 A1 · Device and method for controlling an agent

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0086567 A1
- Document ID: US20260086567A1
- Status: Published application
- Kind: A1
- Application number: 19/332,125
- Filed: 2025-09-18
- Published: 2026-03-26
- Earliest claimed: 2024-09-26
- Applicants: Robert Bosch GmbH
- Inventors: Alessandro Giacomo Bottero; Carlos Enrique Luis Goncalves; Felix Berkenkamp; Jan Peters; Julia Vinogradska
- Web page: https://patentgrep.com/patent/US20260086567A1

## CPC

- G05D 1/60: Intended control result
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/092: Reinforcement learning
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G05D 2101/15: using machine learning, e.g. neural networks
- G05D 2109/10: Land vehicles
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A method for controlling an agent. The method includes determining, for a present state of the agent and a state of an environment of the agent in which the agent should be controlled, a control history indicating a sequence of actions performed by the agent that led to the present state and indicating observations about changes of a state of the agent and/or a state of an environment of the agent, determining an encoding of the control history by supplying the control history to a history encoder comprising a Kalman filter, wherein the encoding is given by a system state estimate determined by the Kalman filter, supplying the encoding to a control policy trained to determine actions from control policy encodings and controlling the agent to perform an action provided by the control policy in response to being supplied with the encoding.

## Claims

### Claim 1

A method for controlling an agent, comprising the following steps:
determining, for a present state of the agent and a state of an environment of the agent in which the agent should be controlled, a control history indicating a sequence of actions performed by the agent that led to the present state and indicating observations about changes of a state of the agent and/or the state of the environment of the agent;
determining an encoding of the control history by supplying the control history to a history encoder including a Kalman filter, wherein the encoding is given by a system state estimate determined by the Kalman filter;
supplying the encoding to a control policy trained to determine actions from control policy encodings; and
controlling the agent to perform an action provided by the control policy in response to being supplied with the encoding.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
training the control policy wherein parameters of the Kalman filter are trained together with the control policy.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
training the control policy using reinforcement learning.

### Claim 4 (depends on 1)

The method of claim 1, wherein the Kalman filter is configured to estimate the system state using a linear structured state space model for the system state and the observations which is given by trainable matrices having diagonal structure.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
parallel processing of multiple control histories.

### Claim 6 (depends on 1)

The method of claim 1, wherein the Kalman filter is configured to repeat, for a control history which indicates a sequence being shorter than a default length, the system state estimate the Kalman filter has determined by an end of the sequence until the Kalman filter has reached a number of estimation iterations corresponding to the default length.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
determining the encoding of the control history by supplying the control history to a first Kalman filter of a sequence of Kalman filters,
supplying system state estimates of each Kalman filter of the sequence, except a last Kalman filter in the sequence, to a next Kalman filter in the sequence, wherein the encoding is given by a system state estimate determined by the last Kalman filter of the sequence.

### Claim 8

A controller configured to control an agent, the controller configured to performing the following steps comprising:
determining, for a present state of the agent and a state of an environment of the agent in which the agent should be controlled, a control history indicating a sequence of actions performed by the agent that led to the present state and indicating observations about changes of a state of the agent and/or the state of the environment of the agent;
determining an encoding of the control history by supplying the control history to a history encoder including a Kalman filter, wherein the encoding is given by a system state estimate determined by the Kalman filter;
supplying the encoding to a control policy trained to determine actions from control policy encodings; and
controlling the agent to perform an action provided by the control policy in response to being supplied with the encoding.

### Claim 9

A non-transitory computer-readable medium on which are stored instructions for controlling an agent, the instructions, when executed by a computer, causing the computer to perform the following steps comprising:
determining, for a present state of the agent and a state of an environment of the agent in which the agent should be controlled, a control history indicating a sequence of actions performed by the agent that led to the present state and indicating observations about changes of a state of the agent and/or the state of the environment of the agent;
determining an encoding of the control history by supplying the control history to a history encoder including a Kalman filter, wherein the encoding is given by a system state estimate determined by the Kalman filter;
supplying the encoding to a control policy trained to determine actions from control policy encodings; and
controlling the agent to perform an action provided by the control policy in response to being supplied with the encoding.
