Transformer framework for trajectory prediction

An apparatus comprising one or more processors configured to implement a method, the processor configured to:

• determine a first cross-attention between a vectorized representation of a road map near a vehicle and information obtained from a rasterized representation of an environment near the vehicle by processing through a first cross-attention stage;
• determine a second cross-attention between a vectorized representation of a vehicle history and information obtained from the rasterized representation by processing through a second cross-attention stage;
• operate a scene encoder on the first cross-attention and the second cross-attention;
• operate a trajectory decoder on an output of the scene encoder;
• generate one or more trajectory predictions by performing one or more queries on the trajectory decoder.

A non-transitory computer-storage medium having process-executable code that, upon execution, causes one or more processor to implement a method, comprising:

• determining a first cross-attention between a vectorized representation of a road map near a vehicle and information obtained from a rasterized representation of an environment near the vehicle by processing through a first cross-attention stage;
• determining a second cross-attention between a vectorized representation of a vehicle history and information obtained from the rasterized representation by processing through a second cross-attention stage;
• operating a scene encoder on the first cross-attention and the second cross-attention;
• operating a trajectory decoder on an output of the scene encoder;
• generating one or more trajectory predictions by performing one or more queries on the trajectory decoder.