Method and system for aligning image data from a vehicle camera

A method of aligning image data from a vehicle camera, the method comprising the steps of:

• obtaining image data from the vehicle camera installed on a vehicle;
• obtaining sensor data from one or more vehicle sensors installed on the vehicle;
• carrying out a yaw/pitch estimation process to obtain a yaw misalignment estimate and a pitch misalignment estimate, wherein the yaw/pitch estimation process includes:
• identifying a plurality of features of a first image frame of the image data and then matching each of the identified features of the first image frame to a corresponding feature of a second image frame of the image data to obtain a plurality of matched features;
• determining a three-dimensional location for each of the plurality of matched features by using a triangulation technique;
• using a three-dimensional to two-dimensional (3D-to-2D) projection mapping technique to determine a relative camera pose based on the three-dimensional locations;
• carrying out the identifying, determining, and using steps a plurality times to obtain a set of relative camera poses;
• and
• determining the yaw misalignment estimate and the pitch misalignment estimate based on the set of relative camera poses and the sensor data;
• after the yaw/pitch estimation process, carrying out a roll estimation process to obtain a roll misalignment estimate;
• and
• applying camera alignment information to the image data to obtain aligned image data, wherein the camera alignment information includes the yaw misalignment estimate, the pitch misalignment estimate, and the roll misalignment estimate.

A vehicle image alignment system, comprising:

• a vehicle camera installed on a vehicle;
• one or more vehicle sensors installed on the vehicle;
• and
• a vehicle video processing module that includes a processor and a non-transitory, computer-readable memory accessible by the processor, wherein the non-transitory, computer-readable memory includes computer instructions;
• wherein when the processor of the vehicle video processing module executes the computer instructions, the vehicle alignment system:
• obtains image data from the vehicle camera;
• obtains sensor data from the one or more vehicle sensors;
• carries out a yaw/pitch estimation process to obtain yaw/pitch camera alignment information that includes a yaw misalignment estimate and a pitch misalignment estimate, wherein the yaw/pitch estimation process uses the image data and the sensor data to determine three-dimensional points of identified features within two or more image frames of the image data when obtaining the yaw/pitch camera alignment information;
• after the yaw/pitch estimation process, carries out a roll estimation process to obtain roll alignment information that includes a roll misalignment estimate, wherein the roll estimation process includes using the yaw/pitch camera alignment information when obtaining the roll alignment information;
• and
• applies camera alignment information to the image data to obtain aligned image data, wherein the camera alignment information includes information based on the yaw/pitch camera alignment information, the roll alignment information, or both.