Systems and Methods for Latent Distribution Modeling for Scene-Consistent Motion Forecasting

A computer-implemented method of training a motion forecasting system, the method comprising:

• obtaining, by a computing system comprising one or more computing devices, a training dataset comprising one or more training examples labeled with ground truth data, the one or more training examples comprising one or more actor features and the ground truth data comprising a ground truth context of the one or more actor features;
• providing, by the computing system, the one or more training examples labeled with ground truth data to a latent encoder model, the latent encoder model configured to produce a first latent distribution in response to receipt of the one or more training examples and the ground truth data;
• providing, by the computing system, the one or more training examples to a latent prior model, the latent prior model configured to produce a second latent distribution in response to receipt of the one or more training examples;
• determining, by the computing system, a training loss based at least in part on the first latent distribution and the second latent distribution;
• and
• backpropagating, by the computing system, the training loss through at least the latent prior model to train at least the latent prior model.

A computer-implemented method of operating an autonomous vehicle, the method comprising:

• obtaining, by the computing system, one or more scene observations;
• providing, by the computing system, the one or more scene observations to a feature extraction model, the feature extraction model configured to produce scene data comprising one or more actor features from the one or more scene observations;
• receiving, by the computing system, the scene data comprising one or more actor features from the feature extraction model;
• providing, by the computing system, the scene data to a latent prior model, the latent prior model configured to generate scene latent data in response to receipt of scene data, the scene latent data comprising one or more latent variables;
• obtaining, by the computing system, the scene latent data from the latent prior model;
• sampling, by the computing system, one or more latent samples from the scene latent data;
• providing, by the computing system, the one or more latent samples to a decoder model, the decoder model configured to decode the latent samples into a motion forecast comprising one or more predicted trajectories of the one or more actor features;
• obtaining, by the computing system, one or more motion forecasts comprising one or more predicted trajectories of the one or more actor features from the decoder model;
• providing, by the computing system, the one or more predicted trajectories to a motion planning model configured to generate a motion plan for an autonomous vehicle based at least in part on the one or more predicted trajectories;
• and
• implementing, by the computing system, the motion plan to control the autonomous vehicle.