Automated graph local constellation (GLC) method of correspondence search for registration of 2-D and 3-D data

A system for registration of two and three-dimensional images, the system comprising:

• a first vehicle system configured to image a 2-dimensional (2-D) image comprising 2-D image points of a geographical region by a camera attached to a first vehicle;
• a second vehicle system configured to scan a 3-dimensional point cloud comprising 3-D cloud image points of the geographical region by a Light Detection and Ranging (LIDAR) system attached to a second vehicle;
• a processor;
• memory storing instructions executable by the processor to perform functions comprising:
• transforming the 2-dimensional (2-D) image of the geographical region via a regional maxima transform or an edge segmenting and boundary filling (ESBF) transform to produce a filtered 2-D image;
• iteratively eroding and opening the filtered 2-D image to produce a processed electro-optical (EO) 2-D image;
• extracting 2-D object shape morphology from the processed EO 2-D image;
• extracting 2-D shape properties from the 2-D object shape morphology;
• generating a height slice of the 3-dimensional (3-D) point cloud comprising LIDAR measurements of the geographical region;
• extracting slice object shape morphology from the height slice resulting in labeled shapes found in the 2-D slice image;
• extracting slice shape properties from the slice object shape morphology;
• constellation matching the 2-D image to the height slice based on the 2-D shape properties and the slice shape properties;
• matching points of the 2-D image and the 3-dimensional point cloud based on the 2-D shape properties and the slice shape properties to produce a plurality of matched points;
• estimating a pose position of the camera based on the matched points to provide an estimated pose position;
• determining a geo-location of the first vehicle based on the estimated pose position of the camera attached to the first vehicle;
• and
• based on the matching points, generating a colorized 3-dimensional point cloud wherein at least some of the 3-D cloud image points of the 3-dimensional point cloud are given a color of an object taken from the 2-D image.

A non-transitory computer readable storage medium comprising computer-executable instructions for registration of two and three-dimensional (3-D) images, the computer-executable instructions comprising:

• capturing a 2-dimensional (2-D) image comprising 2-D image points of a geographical region by a camera attached to a first vehicle;
• transforming the 2-dimensional (2-D) image of the geographical region via a regional maxima transform or an edge segmenting and boundary filling (ESBF) transform to produce a filtered 2-D image;
• iteratively eroding and opening the filtered 2-D image to produce a processed electro-optical (EO) 2-D image;
• extracting 2-D object shape morphology from the processed EO 2-D image;
• extracting 2-D shape properties from the 2-D object shape morphology;
• scanning the geographical region by a Light Detection and Ranging (LIDAR) system attached to a second vehicle to produce a 3-dimensional point cloud comprising 3-D cloud image points thereof;
• generating a height slice of the 3-dimensional point cloud comprising LIDAR measurements of the geographical region;
• extracting slice object shape morphology from the height slice resulting in labeled shapes found in the 2-D slice image;
• extracting slice shape properties from the slice object shape morphology;
• constellation matching the 2-dimensional (2-D) image to the height slice based on the 2-D shape properties and the slice shape properties;
• determining matching points between 2-D image points and 3-D cloud image points to produce a plurality of matched points;
• estimating a pose position of a camera based on the matched points to provide an estimated pose position;
• determining a geo-location of the first vehicle based on the estimated pose position of the camera attached to the first vehicle;
• and
• based on the matching points, generating a colorized 3-dimensional point cloud wherein at least some of the 3-D cloud image points of the 3-dimensional point cloud are given a color of an object taken from the 2-D image.

A method for registration of two and three-dimensional (3-D) images, the method comprising:

• capturing a 2-dimensional (2-D) image comprising 2-D image points of a geographical region by a camera attached to a first vehicle;
• scanning the geographical region by a Light Detection and Ranging (LIDAR) system attached to a second vehicle to produce a 3-dimensional point cloud comprising 3-D cloud image points thereof;
• determining matching points between the 2-D image points and the 3-D cloud image points by;
• finding a plurality of shapes for a plurality of objects;
• identifying the shapes for further extraction to produce identified shapes;
• extracting properties of the identified shapes comprising shape corners and shape dimensions;
• and
• matching a constellation of objects via matching LIDAR object to 2D objects based on the properties of the identified shapes;
• estimating a pose position of the camera attached to the first vehicle based on the matching points to provide an estimated pose position;
• determining a geo-location of the first vehicle based on the estimated pose position of the camera attached to the first vehicle;
• and
• based on the matching points, generating a colorized 3-dimensional point cloud wherein at least some of the 3-D cloud image points of the 3-dimensional point cloud are given a color of an object taken from the 2-D image.