Constructing dynamic environment data with automated annotation

A method for constructing dynamic environment data with an automated annotation, comprising:

• constructing a three-dimensional (3D) model of an environment, using a plurality of sensors located on a machine configured to move through the environment, wherein the plurality of sensors include a first sensor configured to obtain point cloud data of the environment, a second sensor configured to obtain a plurality of two dimensional (2D) images of an object or feature in the environment from different perspectives as the machine moves through the environment, and a third sensor configured to monitor position and orientation of the machine as the machine moves through the environment;
• receiving a first one of the 2D images of the object or feature in the environment, obtained by the second sensor, wherein the first one of the 2D images is an annotated 2D image which includes an annotation identifying the object or feature;
• projecting the annotated 2D images with the annotation onto the 3D model;
• projecting a second one of the 2D images of the object or feature, obtained by the second sensor from a different perspective of the object or feature, onto the 3D model, wherein the second one of the 2D images is a non-annotated 2D image;
• aligning the non-annotated 2D image projected onto the 3D model with the annotated 2D image projected onto the 3D model;
• transferring the annotation from the annotated 2D image projected onto the 3D model to the non-annotated 2D image projected onto the 3D model after the annotated 2D image and the non-annotated 2D image are aligned with one another on the 3D model to convert the non-annotated 2D image to a second annotated 2D image;
• and
• re-projecting the non-annotated 2D image as the second annotated 2D image onto a 2D plane.

A computer-readable storage medium having instructions stored thereon that, when executed by a processing system, perform a method comprising:

• constructing a three-dimensional (3D) model of an environment, using a plurality of sensors located on a machine configured to move through the environment, wherein the plurality of sensors include a first sensor configured to obtain point cloud data of the environment, a second sensor configured to obtain a plurality of two dimensional (2D) images of an object or feature in the environment from different perspectives as the machine moves through the environment, and a third sensor configured to monitor position and orientation of the machine as the machine moves through the environment;
• receiving a first one of the 2D images of the object or feature in the environment, obtained by the second sensor, wherein the first one of the 2D images is an annotated 2D image which includes an annotation identifying the object or feature;
• projecting the annotated 2D images with the annotation onto the 3D model;
• projecting a second one of the 2D images of the object or feature, obtained by the second sensor from a different perspective of the object or feature, onto the 3D model, wherein the second one of the 2D images is a non-annotated 2D image;
• aligning the non-annotated 2D image projected onto the 3D model with the annotated 2D image projected onto the 3D model;
• transferring the annotation from the annotated 2D image projected onto the 3D model to the non-annotated 2D image projected onto the 3D model after the annotated 2D image and the non-annotated 2D image are aligned with one another on the 3D model to convert the non-annotated 2D image to a second annotated 2D image;
• and
• re-projecting the non-annotated 2D image as the second annotated 2D image onto a 2D plane.