Determining respective impacts of agents

A system comprising:

• one or more computers;
• and
• one or more storage devices storing instructions that, when executed by the one or more computers, cause the one or more computers to perform operations for determining respective importance scores for a plurality of agents in a vicinity of an autonomous vehicle navigating through an environment, wherein the respective importance scores characterize a relative impact of each agent on planned trajectories generated by a planning subsystem of the autonomous vehicle, the operations comprising:
• obtaining first state data characterizing a first state of the environment with a first set of agents being located within a vicinity of the autonomous vehicle;
• providing the first state data as input to the planning subsystem;
• obtaining as output from the planning subsystem a first planned trajectory for the autonomous vehicle;
• obtaining second state data characterizing a second state of the environment with a second set of agents being located within a vicinity of the autonomous vehicle, wherein the second set of agents includes (i) all of the agents that are in the first set of agents and (ii) one or more agents that are not in the first set of agents;
• providing the second state data as input to the planning subsystem;
• obtaining as output from the planning subsystem a second planned trajectory for the autonomous vehicle;
• determining a measure of a difference between the first and second planned trajectories;
• and
• determining, based at least in part on the determined measure of difference, corresponding importance score for the one or more agents that are in the second set of agents but not in the first set of agents.

One or more non-transitory computer storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations for determining respective importance scores for a plurality of agents in a vicinity of an autonomous vehicle navigating through an environment, wherein the respective importance scores characterize a relative impact of each agent on planned trajectories generated by a planning subsystem of the autonomous vehicle, the operations comprising:

• obtaining first state data characterizing a first state of the environment with a first set of agents being located within a vicinity of the autonomous vehicle;
• providing the first state data as input to the planning subsystem;
• obtaining as output from the planning subsystem a first planned trajectory for the autonomous vehicle;
• obtaining second state data characterizing a second state of the environment with a second set of agents being located within a vicinity of the autonomous vehicle, wherein the second set of agents includes (i) all of the agents that are in the first set of agents and (ii) one or more agents that are not in the first set of agents;
• providing the second state data as input to the planning subsystem;
• obtaining as output from the planning subsystem a second planned trajectory for the autonomous vehicle;
• determining a measure of a difference between the first and second planned trajectories;
• and
• determining, based at least in part on the determined measure of difference, corresponding importance score for the one or more agents that are in the second set of agents but not in the first set of agents.