Interacting with an unsafe physical environment

A configuration system for configuring an interaction system which interacts with a physical environment, wherein an action of the interaction system in a state of the physical environment results in an updated state of the physical environment according to a transition probability, the configuration system comprising:

• a data interface for accessing data indicating a safe set of state-action pairs known to be safely performable and data indicating an unsafe set of state-action pairs to be avoided when interacting with the physical environment;
• a processor subsystem configured to, while the interaction system interacts with the physical environment, maintain empirical transition probabilities of state-action pairs resulting in updated states, and to iteratively control an interaction of the interaction system with the physical environment by, in an iteration:
• obtain, from the interaction system, data indicating a current state of the physical environment;
• update the safe set of state-action pairs, including:
• estimating an estimated transition probability for each state-action pair of the state-action pairs resulting in updated states based on an empirical transition probability of a similar other state-action pair, and
• including the state-action pair in the safe set of state-action pairs when the state-action pair is not labelled as unsafe and the safe set of state-action pairs can be reached with sufficient probability from the state-action pair based on the estimated transition probability;
• select an action to be performed in the current state of the physical environment from the safe set of state-action pairs;
• providing the action to be performed to the interaction system.

A non-transitory computer-readable medium on which is stored instructions for configuring a system which interacts with a physical environment, wherein an action of the system in a state of the physical environment results in an updated state of the physical environment according to a transition probability, the instructions, when executed by a processor system, causing the processor system to perform the following steps:

• accessing data indicating a safe set of state-action pairs known to be safely performable and data indicating an unsafe set of state-action pairs to be avoided when interacting with the physical environment;
• while the system interacts with the physical environment, maintaining empirical transition probabilities of state-action pairs resulting in updated states;
• and
• iteratively controlling an interaction with the physical environment by, in an iteration:
• obtaining data indicating a current state of the physical environment;
• updating the safe set of state-action pairs, including:
• estimating an estimated transition probability for each state-action pair of the state action pairs resulting in the updated states based on an empirical transition probability of a similar other state-action pair, and
• including the state-action pair in the safe set of state-action pairs when the state-action pair is not labelled as unsafe and the safe set of state-action pairs can be reached with sufficient probability from the state-action pair based on the estimated transition probability;
• selecting an action to be performed in a current state of the physical environment from the safe set of state-action pairs;
• and
• providing the action to be performed to the system.