Device and method for controlling a robot

A method for controlling a technical system, comprising the following steps:

• training a control policy including:
• estimating a variance of a value function which associates:
• (i) a state with a value of the state, or (ii) a pair of state and action with a value of the pair, by solving a Bellman uncertainty equation, wherein, for each of multiple states, a reward function of the Bellman uncertainty equation is set to a difference of a total uncertainty about a mean of a value of a subsequent state following the state and an average aleatoric uncertainty of the value of the subsequent state, and
• biasing the control policy in training towards regions for which the estimation gives a higher variance of the value function than for other regions;
• and
• controlling the technical system according to the trained control policy.

A controller configured to train a control policy, the controller configured to:

• estimate a variance of a value function which associates:
• (i) a state with a value of the state, or (ii) a pair of state and action with a value of the pair, by solving a Bellman uncertainty equation, wherein, for each of multiple states, a reward function of the Bellman uncertainty equation is set to a difference of a total uncertainty about a mean of a value of a subsequent state following the state and an average aleatoric uncertainty of the value of the subsequent state;
• and
• bias the control policy in training towards regions for which the estimation gives a higher variance of the value function than for other regions.

A non-transitory computer-readable medium on which are stored instructions for training a control policy, the instructions, when executed by a computer, causing the computer to perform the following steps:

• estimating a variance of a value function which associates:
• (i) a state with a value of the state, or (ii) a pair of state and action with a value of the pair, by solving a Bellman uncertainty equation, wherein, for each of multiple states, a reward function of the Bellman uncertainty equation is set to a difference of a total uncertainty about a mean of a value of a subsequent state following the state and an average aleatoric uncertainty of the value of the subsequent state;
• and
• biasing the control policy in training towards regions for which the estimation gives a higher variance of the value function than for other regions.