Autonomous vehicle object classifier method and system

A computer-implemented method for object identification, comprising:

• extracting a first set of visual features from a first image of a scene detected by a first sensor;
• extracting a second set of visual features from a second image of the scene detected by a second sensor of a different sensor type than the first sensor;
• concatenating the first set of visual features, the second set of visual features, and a set of bounding box information associated with the first image and the second image;
• determining a number of object features associated with a corresponding number of objects other than an ego-vehicle from the scene and a global feature for the scene;
• receiving ego-vehicle feature information associated with the ego-vehicle;
• receiving the number of object features, the global feature, and the ego-vehicle feature information;
• generating relational features with respect to relationships between each of the number of objects from the scene, the relational features including a mutual influence between each of the number of objects from the scene;
• classifying, by an object classifier, each of the number of objects from the scene as a binary classification of relevant or non-relevant to a behavior of the ego-vehicle, based on the number of object features, the relational features, the global feature, the ego-vehicle feature information, and an intention of the ego-vehicle;
• and
• labeling each of the number of objects from the scene as relevant or non-relevant to the behavior of the ego-vehicle,
• wherein the method further comprises training the object classifier utilizing semi-supervised learning including a labeled dataset and an unlabeled dataset, wherein the unlabeled dataset is annotated with pseudo labels generated from classifying each of the number of objects, the pseudo labels indicating each of the number of objects from the scene as relevant or non-relevant to the behavior of the ego-vehicle.

A system for object identification, comprising:

• a feature extractor:
• extracting a first set of visual features from a first image of a scene detected by a first sensor;
• extracting a second set of visual features from a second image of the scene detected by a second sensor of a different sensor type than the first sensor;
• concatenating the first set of visual features, the second set of visual features, and a set of bounding box information associated with the first image and the second image;
• determining a number of object features associated with a corresponding number of objects other than an ego-vehicle from the scene and a global feature for the scene;
• and
• receiving ego-vehicle feature information associated with the ego-vehicle;
• an object classifier:
• receiving the number of object features, the global feature, and the ego-vehicle feature information;
• generating relational features with respect to relationships between each of the number of objects from the scene, the relational features including a mutual influence between each of the number of objects from the scene;
• classifying each of the number of objects from the scene as a binary classification of relevant or non-relevant to a behavior of the ego-vehicle, based on the number of object features, the relational features, the global feature, the ego-vehicle feature information, and an intention of the ego-vehicle;
• and
• labeling each of the number of objects from the scene as relevant or non-relevant to the behavior of the ego-vehicle;
• a task generator generating a task to be implemented using one or more of the number of object features associated with the corresponding number of objects from the scene classified as relevant to the behavior of the ego-vehicle, while discarding one or more of the number of object features associated with the corresponding number of objects from the scene classified as non-relevant to the behavior of the ego-vehicle;
• and
• an autonomous controller implementing the task by driving one or more vehicle systems to execute the task,
• wherein the object classifier is trained utilizing semi-supervised learning including a labeled dataset and an unlabeled dataset, wherein the unlabeled dataset is annotated with pseudo labels generated from classifying each of the number of objects, the pseudo labels indicating each of the number of objects from the scene as relevant or non-relevant to the behavior of the ego-vehicle.