Method for training autonomous driving model, electronic device, and storage medium

An electronic device, comprising:

• at least one processor;
• and
• a memory communicatively connected to the at least one processor;
• wherein
• the memory stores a computer program executable by the at least one processor to cause the at least one processor to perform a method for training an autonomous driving model;
• wherein the autonomous driving model comprises a driving behavior prediction model and a driving trajectory prediction model, and the method comprises:
• constructing a visual question answering (VQA) network graph based on at least one of an autonomous driving data set or an autonomous driving simulator, wherein the VQA network graph comprises an image frame and a QA network graph corresponding to the image frame, the QA network graph is composed of a plurality of nodes and directed edges among the plurality of nodes, and each of the plurality of nodes carries a question-and-answer (Q&A) pair;
• using the VQA network graph as a sample set and dividing the sample set into a training set and a test set;
• and
• training the driving behavior prediction model and the driving trajectory prediction model based on the training set and evaluating the driving behavior prediction model and the driving trajectory prediction model based on the test set,
• wherein the Q&A pair comprises a question sentence and a real answering sentence, the VQA network graph also carries a real driving behavior and a real driving trajectory corresponding to the image frame, and the real driving behavior comprises a real driving speed and a real steering angle, and
• wherein training the driving behavior prediction model based on the training set comprises:
• traversing the plurality of nodes in sequence according to logical dependencies among the plurality of nodes in the VQA network graph until all the plurality of nodes are traversed;
• in response to that a node of the plurality of nodes traversed has a predecessor node, inputting the image frame and a question sentence of the node into the driving behavior prediction model and outputting a predicted answering sentence of the node, and in response to that a node of the plurality of nodes traversed has a predecessor node, inputting a Q&A pair of the predecessor node and a question sentence of the node into the driving behavior prediction model and outputting a predicted answering sentence of the node;
• inputting Q&A pairs of all the plurality of nodes in the VQA network graph into the driving behavior prediction model and outputting a predicted driving behavior;
• and
• training the driving behavior prediction model based on predicted answering sentences of the plurality of nodes, real answering sentences of the plurality of nodes, the predicted driving behavior, and the real driving behavior.

A non-transitory computer-readable storage medium storing computer instructions configured to, when executed by a processor, cause the processor to perform a method for training an autonomous driving model;

• wherein the autonomous driving model comprises a driving behavior prediction model and a driving trajectory prediction model, and the method comprises:
• constructing a visual question answering (VQA) network graph based on at least one of an autonomous driving data set or an autonomous driving simulator, wherein the VQA network graph comprises an image frame and a QA network graph corresponding to the image frame, the QA network graph is composed of a plurality of nodes and directed edges among the plurality of nodes, and each of the plurality of nodes carries a question-and-answer (Q&A) pair;
• using the VQA network graph as a sample set and dividing the sample set into a training set and a test set;
• and
• training the driving behavior prediction model and the driving trajectory prediction model based on the training set and evaluating the driving behavior prediction model and the driving trajectory prediction model based on the test set,
• wherein the Q&A pair comprises a question sentence and a real answering sentence, the VQA network graph also carries a real driving behavior and a real driving trajectory corresponding to the image frame, and the real driving behavior comprises a real driving speed and a real steering angle, and
• wherein training the driving behavior prediction model based on the training set comprises:
• traversing the plurality of nodes in sequence according to logical dependencies among the plurality of nodes in the VQA network graph until all the plurality of nodes are traversed;
• in response to that a node of the plurality of nodes traversed has a predecessor node, inputting the image frame and a question sentence of the node into the driving behavior prediction model and outputting a predicted answering sentence of the node, and in response to that a node of the plurality of nodes traversed has a predecessor node, inputting a Q&A pair of the predecessor node and a question sentence of the node into the driving behavior prediction model and outputting a predicted answering sentence of the node;
• inputting Q&A pairs of all the plurality of nodes in the VQA network graph into the driving behavior prediction model and outputting a predicted driving behavior;
• and
• training the driving behavior prediction model based on predicted answering sentences of the plurality of nodes, real answering sentences of the plurality of nodes, the predicted driving behavior, and the real driving behavior.