Vision-Based Perception System

A method of training for object recognition a first neural network of a vision-based perception system, comprising:

• obtaining a temporal sequence of point cloud data sets by a Light Detection and Ranging (LIDAR) device of the vision-based perception system representing an environment of the vision-based perception system;
• capturing a temporal sequence of images of the environment by a camera device of the vision-based perception system;
• determining an unrecognized object in the temporal sequence of images by a processing unit of the vision-based perception system;
• determining a bounding box for the determined unrecognized object based on at least one of the point cloud data sets by the processing unit;
• obtaining a neural network representation of the unrecognized object based on the temporal sequence of images and the determined bounding box by means of a second neural network of the vision-based perception system;
• and
• training the first neural network for recognition of the unrecognized object based on the obtained neural network representation of the unrecognized object.

A computer program product stored on a non-transitory computer readable medium, comprising instructions to perform steps comprising:

• obtaining a temporal sequence of point cloud data sets by a Light Detection and Ranging (LIDAR) device of a vision-based perception system representing an environment of the vision-based perception system;
• capturing a temporal sequence of images of an environment by a camera device of the vision-based perception system;
• determining an unrecognized object in the temporal sequence of images by a processing unit of the vision-based perception system;
• determining a bounding box for the determined unrecognized object based on at least one of the point cloud data sets by the processing unit;
• obtaining a neural network representation of the unrecognized object based on the temporal sequence of images and the determined bounding box by means of a second neural network of the vision-based perception system;
• and
• training the first neural network for recognition of the unrecognized object based on the obtained neural network representation of the unrecognized object.