Vehicle trajectory prediction model with semantic map and lstm

A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations to predict a trajectory of a target obstacle detected by an autonomous driving vehicles (ADV), the operations comprising:

• in response to an image of a driving environment around the target obstacle, processing the image by a first neural network (NN) to learn semantic map features of the image;
• embedding features of the target obstacle into a low-dimensional vector;
• processing the low-dimensional vector and most recent dynamic features of the target obstacle by a second NN to learn updated dynamic features of the target obstacle;
• processing the updated dynamic features of the target obstacle and the semantic map features of the image map by a third NN to generate a next predicted position of the target obstacle, the next predicted position including predicted features;
• and
• repeating embedding the predicted features of the next predicted position, processing by the second NN and processing by the third NN until all predicted positions of a prediction interval are generated.

A data processing system, comprising:

• a processor;
• and
• a memory coupled to the processor to store instructions, which when executed by the processor, cause the processor to predict a trajectory of a target obstacle detected by an autonomous driving vehicles (ADV), the operations comprising:
• in response to an image of a driving environment around the target obstacle, processing the image by a first neural network (NN) to learn semantic map features of the image,
• embedding features of the target obstacle into a low-dimensional vector;
• processing the low-dimensional vector and most recent dynamic features of the target obstacle by a second NN to learn updated dynamic features of the target obstacle,
• processing the updated dynamic features of the target obstacle and the semantic map features of the image map by a third NN to generate a next predicted position of the target obstacle, the next predicted position including predicted features, and
• repeating embedding the predicted features of the next predicted position, processing by the second NN and processing by the third NN until all predicted positions of a prediction interval are generated.