Enhancing perception range of surround view cameras for multi-modal learning

A system for enhancing perception range of a computer vision system, the system comprising:

• a memory for storing a plurality of panoramic pictures;
• and
• processing circuitry in communication with the memory, wherein the processing circuitry is configured to:
• obtain the plurality of panoramic pictures related to an intended driving trajectory of a vehicle;
• convert each of the plurality of panoramic pictures into a plurality of perspective pictures;
• apply one or more machine learning models to the plurality of perspective pictures to generate information related to a plurality of static road objects;
• generate a 3D point cloud corresponding to the intended driving trajectory of the vehicle, including performing a 3D reconstruction on the plurality of perspective pictures;
• extract, using the 3D point cloud, locations of the plurality of detected static road objects;
• and
• operate a vehicle assistance system, while the vehicle is moving along the intended driving trajectory, based on the information related to the plurality of static road objects and the locations of the plurality of static road objects.

Non-transitory computer-readable storage media having instructions encoded thereon, the instructions configured to cause processing circuitry to:

• obtain a plurality of panoramic pictures related to an intended driving trajectory of a vehicle;
• convert each of the plurality of panoramic pictures into a plurality of perspective pictures;
• apply one or more machine learning models to the plurality of perspective pictures to generate information related to a plurality of static road objects;
• generate a 3D point cloud corresponding to the intended driving trajectory of the vehicle, including performing a 3D reconstruction on the plurality of perspective pictures;
• extract, using the 3D point cloud, locations of the plurality of detected static road objects;
• and
• operate a vehicle assistance system, while the vehicle is moving along the intended driving trajectory, based on the information related to the plurality of static road objects and the locations of the plurality of static road objects.