Method, apparatus and system for auto-labeling

A system for auto-labeling dynamic vision sensor (DVS) frames comprising:

• a pair of cameras configured to generate a pair of camera frames within an interval, wherein the pair of cameras includes a left camera and a right camera;
• a DVS configured to generate at least one DVS frame within the interval, wherein the DVS is rigidly combined with the left camera and the right camera such that the DVS and the pair of cameras share a common world coordinate system;
• and
• a computing device comprising a processor and a memory unit storing instructions executable by the processor to:
• calculate a disparity frame based on the pair of camera frames, and obtain 3D information of the pair of camera frames based on the calculated disparity frame;
• determine an object area for auto-labeling using a deep learning model;
• obtain 3D points based on the obtained 3D information and the determined object area, and re-project the 3D points towards the at least one DVS frame to generate re-projected points on the at least one DVS frame;
• and
• generate at least one auto-labeled result on the at least one DVS frame by combining the re-projected points on the at least one DVS frame.

An apparatus for auto-labeling dynamic vision sensor (DVS) frames comprising:

• a computing device comprising a processor and a memory unit instructions executable by the processor to:
• receive a pair of camera frames generated by a pair of cameras within an interval, and receive at least one DVS frame generated by a DVS within the interval, wherein the pair of cameras includes a left camera and a right camera, and wherein the DVS is rigidly combined with the left camera and the right camera such that the DVS and the pair of cameras share a common world coordinate system;
• calculate a disparity frame based on the pair of camera frames, and obtain 3D information of the pair of camera frames based on the calculated disparity frame;
• determine an object area for auto-labeling using a deep learning model;
• obtain 3D points based on the obtained 3D information and the determined object area, and re-project the 3D points towards the at least one DVS frame to generate re-projected points on the at least one DVS frame;
• and
• generate at least one auto-labeled result on the at least one DVS frame by combining the re-projected points on the at least one DVS frame.