Systems and methods for generating motion forecast data for a plurality of actors with respect to an autonomous vehicle

A computer-implement method for forecasting actor motion, the method comprising:

• inputting, by a computing system comprising one or more computing devices, an input representation that describes sensor data into an object detection model that is configured to receive the input representation that describes the sensor data, and in response to receipt of the input representation, output object detection data describing features of a plurality of actors relative to an autonomous vehicle;
• receiving, by the computing system and as an output of the object detection model, the object detection data describing the features of the plurality of the actors relative to the autonomous vehicle:
• generating, by the computing system, an input sequence using an interaction model;
• analyzing, by the computing system, the input sequence using an attention model to generate attentional weights;
• receiving, by the computing system, the attentional weight and input sequence using a context aggregation model, and in response to receipt of the attentional weights and the input sequence, generate an attention embedding with respect to the plurality of actors;
• inputting, by the computing system, the attention embedding into a recurrent model that is configured to receive the attention embedding, and in response to receipt of the attention embedding, generate motion forecast data with respect to the plurality of actors, the motion forecast data describing respective trajectories for the plurality of actors;
• and
• determining, by the computing system, respective trajectories for the plurality of actors based on motion forecast data received as an output of a recurrent model, the recurrent model being configured to receive the attention embedding, and in response to receipt of the attention embedding, generate motion forecast data with respect to the plurality of actors.

A computer-implement method for training one or more machine-learned systems, the method comprising:

• inputting, by a computing system comprising one or more computing devices, an input representation that describes sensor data into an object detection model that is configured to receive the input representation that describes the sensor data, and in response to receipt of the input representation, output object detection data describing features of a plurality of actors relative to an autonomous vehicle;
• receiving, by the computing system and as an output of the object detection model, the object detection data describing the features of the plurality of the actors relative to the autonomous vehicle;
• generating, by the computing system, an input sequence using an interaction model;
• analyzing, by the computing system, the input sequence using an attention model to generate attentional weights,
• receiving, by the computing system, the attentional weight and input sequence using a context aggregation model, and in response to receipt of the attentional weights and the input sequence, generate an attention embedding with respect to the plurality of actors;
• inputting, by the computing system, the attention embedding into a recurrent model that is configured to receive the attention embedding, and in response to receipt of the attention embedding, generate motion forecast data with respect to the plurality of actors, the motion forecast data describing respective trajectories for the plurality of actors;
• determining, by the computing system, respective trajectories for the plurality of actors based on motion forecast data received as an output of a recurrent model, the recurrent model being configured to receive the attention embedding, and in response to receipt of the attention embedding, generate motion forecast data with respect to the plurality of actors;
• and
• updating, by the computing system, one or more parameters of at least one of the object detection model, the interaction model, or the recurrent model.