Joint rolling shutter image stitching and rectification

A system for applying rolling shutter (RS)-aware spatially varying differential homography fields for simultaneous RS distortion removal and image stitching, the system comprising:

• a memory;
• and
• a processor in communication with the memory, wherein the processor runs program code to:
• input two consecutive frames including RS distortions from a video stream;
• perform keypoint detection and matching to extract correspondences between the two consecutive frames;
• feed the correspondences between the two consecutive frames into an RS-aware differential homography estimation component to filter out outlier correspondences;
• send inlier correspondences to an RS-aware spatially varying differential homography field estimation component to compute an RS-aware spatially varying differential homography field;
• and
• use the RS-aware spatially varying differential homography field in an RS stitching and correction component to produce stitched images with removal of the RS distortions.

A non-transitory computer-readable storage medium comprising a computer-readable program for applying rolling shutter (RS)-aware spatially varying differential homography fields for simultaneous RS distortion removal and image stitching, wherein the computer-readable program when executed on a computer causes the computer to perform the steps of:

• inputting two consecutive frames including RS distortions from a video stream;
• performing keypoint detection and matching to extract correspondences between the two consecutive frames;
• feeding the correspondences between the two consecutive frames into an RS-aware differential homography estimation component to filter out outlier correspondences;
• sending inlier correspondences to an RS-aware spatially varying differential homography field estimation component to compute an RS-aware spatially varying differential homography field;
• and
• using the RS-aware spatially varying differential homography field in an RS stitching and correction component to produce stitched images with removal of the RS distortions.