Spatio-temporal pose/object database

A system comprising one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:

• determining a current geolocation of an agent within an environment, wherein the current geolocation corresponds to a voxel of a voxel grid imposed on the environment, wherein the voxel represents a respective bounded three-dimensional region of the environment, wherein the agent is a simulated agent in a simulation of a real-world environment, and wherein the simulation of the real-world environment is for evaluating control software for an autonomous vehicle;
• obtaining historical data for the voxel corresponding to the current geolocation of the agent from a database that maintains historical data for a plurality of voxels of the voxel grid, the historical data for each voxel comprising a respective trajectory for each of one or more objects that were detected at the voxel by sensors of a vehicle navigating through the environment, wherein each trajectory comprises one or more voxels at which the respective object was detected;
• generating a future trajectory for the agent based at least in part on the historical data, comprising:
• identifying, from the historical data, a plurality of trajectories, each trajectory corresponding to a respective agent and including the current geolocation for the agent;
• identifying, from the historical data and for each of the plurality of identified trajectories, a respective observed next geolocation that follows the current geolocation in the identified trajectory;
• computing a distribution over the plurality of observed next geolocations that assigns a respective probability to each of the plurality of observed next geolocations, wherein the observed next geolocation corresponds to another voxel of the established voxel grid;
• and
• selecting an observed next geolocation from the plurality of observed next geolocations as a next location in the future trajectory based on the respective probabilities assigned to the observed next geolocations;
• and
• including, in the simulation, the simulated agent following the future trajectory.

One or more non-transitory computer storage media encoded with computer program instructions that when executed by a plurality of computers cause the plurality of computers to perform operations comprising:

• determining a current geolocation of an agent within an environment, wherein the current geolocation corresponds to a voxel of a voxel grid imposed on the environment, wherein the voxel represents a respective bounded three-dimensional region of the environment, wherein the agent is a simulated agent in a simulation of a real-world environment, and wherein the simulation of the real-world environment is for evaluating control software for an autonomous vehicle;
• obtaining historical data for the voxel corresponding to the current geolocation of the agent from a database that maintains historical data for a plurality of voxels of the voxel grid, the historical data for each voxel comprising a respective trajectory for each of one or more objects that were detected at the voxel by sensors of a vehicle navigating through the environment, wherein each trajectory comprises one or more voxels at which the respective object was detected;
• and
• generating a future trajectory for the agent based at least in part on the historical data, comprising:
• identifying, from the historical data, a plurality of trajectories, each trajectory corresponding to a respective agent and including the current geolocation for the agent;
• identifying, from the historical data and for each of the plurality of identified trajectories, a respective observed next geolocation that follows the current geolocation in the identified trajectory, wherein the observed next geolocation corresponds to another voxel of the established voxel grid;
• selecting an observed next geolocation from the plurality of observed next geolocations as a next location in the future trajectory based on the respective probabilities assigned to the observed next geolocations;
• and
• including, in the simulation, the simulated agent following the future trajectory.