Intelligent vehicle systems and control logic for surround view augmentation with object model recognition

A method for controlling operation of a motor vehicle with a sensor array including a network of cameras mounted at discrete locations on the motor vehicle, the method comprising:

• receiving, via a system controller from the sensor array, camera data indicative of a camera image with a target object from a camera perspective of one of the cameras;
• analyzing, via the system controller using an object recognition module, the camera image to identify characteristics of the target object and classify the characteristics to a corresponding one of multiple model collection sets associated with a type of the target object;
• retrieving, via the system controller from a memory-stored object library, a 3D object model universal to the corresponding model collection set associated with the type of the target object;
• deriving, via the system controller using a depth inference module, a respective depth for each pixel in the camera image, the deriving including:
• receiving supplemental camera data indicative of overlapping camera images with the target object from multiple camera perspectives of multiple ones of the cameras;
• and
• processing the supplemental camera data through a neural network trained to output depth data and semantic segmentation data using a loss function that combines multiple loss terms including a semantic segmentation loss term and a panoramic loss term, the panoramic loss term including a similarity measure for overlapping patches of the camera data that each corresponds to a region of overlapping fields of view of the multiple ones of the cameras;
• and
• generating a new image from a chosen viewpoint using camera data from at least one of the cameras and replacing the target object in the new image with the 3D object model positioned in a distinct orientation.

A method for controlling operation of a motor vehicle with a sensor array including a network of cameras mounted at discrete locations on the motor vehicle, the method comprising:

• receiving, via a system controller from the sensor array, camera data indicative of a camera image with a target object from a camera perspective of one of the cameras;
• analyzing, via the system controller using an object recognition module, the camera image to identify characteristics of the target object and classify the characteristics to a corresponding one of multiple model collection sets associated with a type of the target object;
• retrieving, via the system controller from a memory-stored object library, a 3D object model universal to the corresponding model collection set associated with the type of the target object;
• generating, via the system controller using an epipolar reprojection module, a image from an alternative perspective of a virtual camera, wherein generating the image includes:
• receiving a real-time orientation of the one of the cameras when capturing the camera image with the target object;
• receiving a desired orientation of the virtual camera for presenting the target object from the alternative perspective in the virtual image;
• defining an epipolar geometry between the real-time orientation of the camera and the desired orientation of the virtual camera;
• generating the virtual image based on a calculated epipolar relation between the real-time orientation and the desired orientation;
• and
• generating, using the virtual image, a new image from a chosen viewpoint using camera data from at least one of the cameras and replacing the target object in the new image with the 3D object model positioned in a distinct orientation.