Systems and methods for training a neural network for generating a reasoning statement

A computer-implemented method for training a neural network for generating a reasoning statement, comprising:

• receiving sensor data having a number of frames imaging at least one roadway environment from a perspective of an ego agent;
• identifying a plurality of captured objects in the at least one roadway environment from the number of frames;
• receiving a set of ranking classifications for a captured object, having an object type, of the plurality of captured objects, wherein a ranking classification of the set of ranking classifications includes an annotator reasoning statement and an applied attribute of a predetermined group of attributes including a plurality of importance attributes in an importance category and an unimportance attribute in an unimportance category, wherein the annotator reasoning statement is a natural language explanation for the applied attribute, and wherein each ranking classification is received from a different source of a plurality of sources;
• generating a training dataset for the object type including the annotator reasoning statements of the set of ranking classifications that include the applied attribute from the plurality of importance attributes in the importance category;
• and
• training the neural network to generate a generated reasoning statement based on the training dataset in response to a training agent detecting a detected object of the object type.

A non-transitory computer readable storage medium storing instructions that when executed by a computer having a processor to perform a method for training a neural network for generating a reasoning statement, the method comprising:

• receiving sensor data having a number of frames imaging at least one roadway environment from a perspective of an ego agent;
• identifying a plurality of captured objects in the at least one roadway environment from the number of frames;
• receiving a set of ranking classifications for a captured object, having an object type, of the plurality of captured objects, wherein a ranking classification of the set of ranking classifications includes an annotator reasoning statement and an applied attribute of a predetermined group of attributes including a plurality of importance attributes in an importance category and an unimportance attribute in an unimportance category, wherein the annotator reasoning statement is a natural language explanation for the applied attribute, and wherein each ranking classification is received from a different source of a plurality of sources;
• generating a training dataset for the object type including the annotator reasoning statements of the set of ranking classifications that include the applied attribute from the plurality of importance attributes in the importance category;
• and
• training the neural network to generate a generated reasoning statement based on the training dataset in response to a training agent detecting a detected object of the object type.