State estimation and localization for ROV-based structural inspection

A system for localizing an underwater remotely controlled vehicle moving within a structure filled at least partially with water, comprising:

• a map of the structure, the map indicating locations of one or more landmarks defined on the structure;
• a pan-tilt-zoom camera, disposed in the water at a stationary position with respect to the structure;
• software, executing on a processor, the software iterating a loop performing the functions of:
• panning, tilting and zooming the camera to follow a plurality of fiducials mounted on the vehicle;
• collecting an image from the camera;
• estimating an angular velocity of the camera about a vector comprising a combination of the pan and tilt axes of the camera;
• predicting a future state of the system based on a current state of the system and the estimate of the angular velocity;
• calculating a correction to the predicted state of the system based on detecting in the collected image, the plurality of fiducials on the vehicle and one or more of the landmarks indicated on the map of the structure;
• applying the correction to the predicted state of the system;
• and
• using the corrected predicted state as a previous state for a next iteration of the loop;
• wherein the predicted state of the system includes at least a position and an orientation of the vehicle with respect to the structure and current pan, tilt and zoom settings of the camera.