Method for rendering 2D and 3D data within a 3D virtual environment

One or more tangible non-transitory computer-readable storage media storing computer-executable instructions for performing a computer process on a computing system, the computer process comprising:

• obtaining a 2D color image of a scene around a vehicle;
• obtaining a 3D point cloud of the scene around the vehicle;
• detecting a cluster of points in the 3D point cloud approximating a plane, the cluster of points representing a continuous surface;
• identifying a cluster of color pixels in the 2D color image, the cluster of pixels depicting the continuous surface;
• defining a set of synthetic 3D color points in the 3D point cloud by projecting the cluster of color pixels onto the 3D point cloud relative to the plane;
• compiling the cluster of points and the set of synthetic 3D color points into a 3D frame;
• detecting characteristics of an object based on the 3D frame, the object including the continuous surface;
• and
• electing a next navigational action for execution by the vehicle based on the characteristics of the object.

A system for augmenting 3D depth map data with 2D color image data comprising:

• a 3D depth sensor arranged on a vehicle, the 3D depth sensor capturing a 3D point cloud, a cluster of points in the 3D point cloud approximating a plane representing a continuous surface;
• a 2D color camera arranged on the vehicle, the 3D depth sensor and the 2D color camera defining intersecting fields of view of a scene around the vehicle, the 2D color camera capturing a 2D color image, a cluster of color pixels in the 2D color image depicting the continuous surface;
• and
• at least one processor defining a set of synthetic 3D color points in the 3D point cloud by projecting the cluster of color pixels onto the 3D point cloud relative to the plane, the cluster of points and the set of synthetic 3D color points representing the continuous surface.