Action selection using interaction history graphs

A system comprising one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the one or more computers to perform operations for selecting actions to be performed by an agent interacting with an environment, the operations comprising:

• obtaining data defining an interaction history graph of nodes and edges that represents an interaction history of the agent with the environment, wherein:
• each node in the interaction history graph represents a possible state of the environment,
• each edge in the interaction history graph connects a respective pair of nodes in the interaction history graph, and
• an edge in the interaction history graph connects a pair of nodes in the interaction history graph only if the state of the environment can transition from one of the nodes in the pair of nodes to the other node in the pair of nodes;
• processing data defining the interaction history graph representing the interaction history of the agent with the environment using a graph neural network to generate an encoded representation of the interaction history graph, comprising:
• obtaining a respective encoded representation of each node in the interaction history graph;
• iteratively updating the encoded representations of the nodes in the interaction history graph, over a plurality of update iterations, in accordance with values of a set of graph neural network parameters;
• and
• generating the encoded representation of the interaction history graph by combining the encoded representations of the nodes in the interaction history graph;
• processing an input based on the encoded representation of the interaction history graph using an action selection neural network, in accordance with current values of action selection neural network parameters, to generate an action selection output;
• and
• selecting an action from a plurality of possible actions to be performed by the agent using the action selection output generated by the action selection neural network.

One or more non-transitory computer storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations for selecting actions to be performed by an agent interacting with an environment, the operations comprising:

• obtaining data defining an interaction history graph of nodes and edges that represents an interaction history of the agent with the environment, wherein:
• each node in the interaction history graph represents a possible state of the environment,
• each edge in the interaction history graph connects a respective pair of nodes in the interaction history graph, and
• an edge in the interaction history graph connects a pair of nodes in the interaction history graph only if the state of the environment can transition from one of the nodes in the pair of nodes to the other node in the pair of nodes;
• processing data defining the interaction history graph representing the interaction history of the agent with the environment using a graph neural network to generate an encoded representation of the interaction history graph, comprising:
• obtaining a respective encoded representation of each node in the interaction history graph;
• iteratively updating the encoded representations of the nodes in the interaction history graph, over a plurality of update iterations, in accordance with values of a set of graph neural network parameters;
• and
• generating the encoded representation of the interaction history graph by combining the encoded representations of the nodes in the interaction history graph;
• processing an input based on the encoded representation of the interaction history graph using an action selection neural network, in accordance with current values of action selection neural network parameters, to generate an action selection output;
• and
• selecting an action from a plurality of possible actions to be performed by the agent using the action selection output generated by the action selection neural network.