Robot vision in autonomous underwater vehicles using the color shift in underwater imaging

A method of generating a point cloud in a coordinate system where the point cloud represents an object comprising:

• obtaining one or more digital cameras where each of the one or more digital cameras is programmed to capture an image and generate a pixmap of the image, where the pixmap comprises a plurality of pixels, where each pixel comprises a pixel value referenced to a color table, where the color table represents a plurality of colors where each color is referenced to at least a first primary color by a first color space coordinate and a second primary color by a second color space coordinate, where the first primary color represents a first wavelength bandwidth of light wavelengths and the second primary color represents a second wavelength bandwidth of light wavelengths;
• capturing an unfiltered image of the object and generating an unfiltered pixmap comprising unfiltered pixels using an individual camera comprising the one or more digital cameras;
• capturing a filtered image of the object and generating a filtered pixmap comprising filtered pixels using a specific digital camera comprising the one or more cameras and a camera filter, where the specific digital camera has a field of view, and where the camera filter has a Full Width at Half Maximum (FWHM) bandwidth where a specific light wavelength is within the first wavelength bandwidth of light wavelengths and within the FWHM bandwidth of the camera filter, and where the camera filter is within the field of view of the specific digital camera and between the specific digital camera and the object;
• generating the point cloud in the coordinate system comprising at least a first dimension, a second dimension, and a third dimension by communicating the unfiltered pixmap and the filtered pixmap from the one or more digital cameras to a processor, where the processor is programmed to perform steps comprising:
• pixel matching the unfiltered pixmap and the filtered pixmap and generating a plurality of match pairs, where each match pair associates a specific unfiltered pixel comprising the unfiltered pixmap with a specific filtered pixel comprising the filtered pixmap, and where the each match pair is assigned an (x,y) pair coordinate based on a location of the specific unfiltered pixel in the unfiltered pixmap and a location of the specific filtered pixel in the filtered pixmap, thereby generating the plurality of match pairs;
• generating the point cloud in the coordinate system by, for each match pair comprising the plurality of match pairs, performing steps comprising:
• extracting an unfiltered first color space coordinate from the unfiltered pixel of the each matched pair, where the unfiltered first color space coordinate is referenced to the first primary color;
• extracting a filtered first color space coordinate from the filtered pixel of the each matched pair, where the filtered first color space coordinate is referenced to the first primary color;
• determining an image distance for the each match pair using a difference between a logarithm of a first term and a logarithm of a second term, where the first term comprises the unfiltered first color space coordinate of the each match pair and the second term comprises the filtered first color space coordinate of the each match pair;
• and
• generating a data point for the each match pair in the coordinate system by using the (x,y) pair coordinate assigned to the each match pair to define a coordinate in the first dimension and a coordinate in the second dimension and using the image distance for the each match pair to define a coordinate in the third dimension, thereby generating the point cloud in the coordinate system where the point cloud represents the object.

A robot vision apparatus comprising:

• an imaging apparatus comprising:
• one or more digital cameras comprising a specific camera and the specific camera having a field-of-view, where each of the one or more digital cameras is programmed to capture an image and generate a pixmap of the image, where the pixmap comprises a plurality of pixels, where each pixel comprises a pixel value referenced to a color table, where the color table represents a plurality of colors where each color is referenced to at least a first primary color by a first color space coordinate and a second primary color by a second color space coordinate, and where the first primary color represents a first wavelength bandwidth of light wavelengths and the second primary color represents a second wavelength bandwidth of light wavelengths, and where the first wavelength bandwidth comprises a first center wavelength and the second wavelength bandwidth comprises a second center wavelength, and the first center wavelength is greater than the second center wavelength;
• a camera filter having a Full Width at Half Maximum (FWHM) bandwidth where a specific light wavelength is within the first wavelength bandwidth of light wavelengths and within the FWHM bandwidth of the camera filter;
• a filter mechanism mechanically configured to position the camera filter in at least a filtering position, where the filtering position is within the field of view of the specific camera;
• a processor in data communication with the imaging apparatus and the processor programmed to generate a point cloud in a coordinate system by performing steps comprising:
• generating an unfiltered pixmap comprising unfiltered pixels and a filtered pixmap comprising filtered pixels by performing steps comprising:
• directing the imaging apparatus to capture an unfiltered image and generate the unfiltered pixmap using an individual camera comprising the one or more cameras, where the individual camera has an individual field of view and where the filter is not within the individual field of view of the individual camera, thereby generating the unfiltered pixmap comprising the unfiltered pixels;
• directing the imaging apparatus to capture a filtered image and generate the filtered pixmap using the specific camera and with the camera filter in the filtering position, thereby generating the filtered pixmap comprising the filtered pixels;
• and
• retrieving the unfiltered pixmap comprising the unfiltered pixels and the filtered pixmap comprising the filtered pixels from the imaging apparatus;
• pixel matching the unfiltered pixmap and the filtered pixmap and generating a plurality of match pairs, where each match pair associates a specific unfiltered pixel comprising the unfiltered pixmap with a specific filtered pixel comprising the filtered pixmap, and where the each match pair is assigned an (x,y) pair coordinate based on a location of the specific unfiltered pixel in the unfiltered pixmap and a location of the specific filtered pixel in the filtered pixmap, thereby generating the plurality of match pairs;
• generating the point cloud in the coordinate system using the plurality of match pairs, where the coordinate system has at least a first dimension, a second dimension, and a third dimension, by, for each match pair comprising the plurality of match pairs, performing steps comprising:
• extracting an unfiltered first color space coordinate from the unfiltered pixel of the each matched pair, where the unfiltered first color space coordinate is referenced to the first primary color;
• extracting a filtered first color space coordinate from the filtered pixel of the each matched pair, where the filtered first color space coordinate is referenced to the first primary color;
• determining an image distance for the each match pair using a difference between a logarithm of a first term and a logarithm of a second term, where the first term comprises the unfiltered first color space coordinate of the each match pair and the second term comprises the filtered first color space coordinate of the each match pair, and where the difference between the logarithm of the first term and the logarithm of the second term divided by a value ΔR is greater than 0.75 and less than 1.25, where the value ΔR is equal to a scaling factor multiplied by a difference between a logarithm of the unfiltered first color space coordinate for the each match pair and a logarithm of the filtered first color space coordinate for the each match pair, where the scaling factor is a constant value for all match pairs comprising the plurality of match pairs;
• and
• generating a data point for the each match pair in the coordinate system by using the (x,y) pair coordinate assigned to the each match pair to define a coordinate in the first dimension and a coordinate in the second dimension and using the image distance for the each match pair to define a coordinate in the third dimension, thereby generating the point cloud in the coordinate system.