Sensor fusion based perceptually enhanced surround view

A non-transitory program storage device comprising instructions stored thereon to cause one or more processors to:

• obtain a first location of a vehicle, the vehicle having two or more cameras disposed about the vehicle, and wherein each camera is associated with a physical camera pose indicating where each camera is located with respect to the vehicle;
• receive, from a first camera, a first image of a first area in a first field of view of the first camera;
• associate the first image with the first location of the vehicle when the first image was captured;
• store the first image of the first area in an image buffer;
• obtain a second location of a vehicle;
• capture, by the first camera, a second image of a second area that at least partially overlaps with the first area;
• associate the second image with the second location of the vehicle when the second image was captured;
• store the second image of the second area in the image buffer when a distance of the second location of the vehicle from the first area is larger than a threshold distance, the first image and the second image in the image buffer are stored in temporal order;
• obtain a third location of the vehicle after the vehicle has moved in a direction so that the first area is in an expected second field of view of a second camera, wherein the second camera is not capturing images;
• determine a temporal camera pose based on a first physical camera pose of the first camera, a second physical camera pose of the second camera, and the third location of the vehicle;
• and
• render a view of the first area based on the temporal camera pose, the first image stored in the image buffer, and the second image stored in the image buffer.

A system for rendering a view around a vehicle, the system comprising:

• two or more cameras disposed about the vehicle, wherein each camera is associated with a physical camera pose indicating where each camera is located with respect to the vehicle;
• a memory;
• and
• one or more processors operatively coupled to the memory, and the one or more cameras, wherein the one or more processors are configured to execute non-transitory instructions causing the one or more processors to:
• obtaining a first location of the vehicle;
• capture, by a first camera, a first image of a first area in a first field of view of the first camera;
• associate the first image with the first location of the vehicle when the first image was captured;
• store the first image of the first area in an image buffer;
• obtain a second location of a vehicle;
• capture, by the first camera, a second image of a second area that at least partially overlaps with the first area;
• associate the second image with the second location of the vehicle when the second image was captured;
• store the second image of the second area in the image buffer when a distance of the second location of the vehicle from the first area is larger than a threshold distance, the first image and the second image in the image buffer are stored in temporal order;
• obtain a third location of the vehicle after the vehicle has moved in a direction so that the first area is in an expected second field of view of a second camera, wherein the second camera is not capturing images;
• determine a temporal camera pose based on a first physical camera pose of the first camera, a second physical camera pose of the second camera, and the third location of the vehicle;
• and
• render a view of the first area based on the temporal camera pose, the first image stored in the image buffer, and the second image stored in the image buffer.