System and method for providing object-level driver attention reasoning with a graph convolution network

A system for providing object-level attention reasoning comprising:

• a memory storing instructions when executed by a processor cause the processor to:
• receive images associated with a plurality of image clips of a surrounding environment of a vehicle;
• analyze the images and determining a plurality of relevant objects that are included within the surrounding environment that influence an operation of the vehicle;
• input data associated with the plurality of relevant objects to a graph convolution network to determine importance scores for each of the plurality of relevant objects;
• and
• determine a plurality of top relevant objects based on the importance scores that are to be accounted for to autonomously control the vehicle to be operated to complete attention reasoning within the surrounding environment of a vehicle, wherein
• analyzing the images and determining the plurality of relevant objects includes determining a target image frame from the plurality of image clips and inputting the target image frame to a regional proposal network to output object-ness scores, and
• the object-ness scores are associated with a likelihood that the plurality of relevant objects included within the plurality of image clips influence the operation of the vehicle.

A non-transitory computer readable storage medium storing instructions that when executed by a computer, which includes a processor perform a method, the method comprising:

• receiving images associated with a plurality of image clips of a surrounding environment of a vehicle;
• analyzing the images and determining a plurality of relevant objects that are included within the surrounding environment that influence an operation of the vehicle;
• inputting data associated with the plurality of relevant objects to a graph convolution network to determine importance scores for each of the plurality of relevant objects;
• and
• determining a plurality of top relevant objects based on the importance scores that are to be accounted for to autonomously control the vehicle to be operated to complete attention reasoning within the surrounding environment of a vehicle, wherein
• determining the importance scores includes inputting updated node features of each object node output by the graph convolution network to a multi-layer perceptron neural network to determine the importance scores for each of the plurality of relevant objects.