Joint rolling shutter correction and image deblurring

A non-transitory computer-readable storage medium comprising a computer-readable program for jointly removing rolling shutter (RS) distortions and blur artifacts in a single input RS and blurred image, wherein the computer-readable program when executed on a computer causes the computer to perform the steps of:

• generating a plurality of RS blurred images from a camera;
• synthesizing RS blurred images from a set of GS sharp images, corresponding GS sharp depth maps, and synthesized RS camera motions by employing a structure-and-motion-aware RS distortion and blur rendering module to generate training data to train a single-view joint RS correction and deblurring convolutional neural network (CNN);
• and
• predicting an RS rectified and deblurred image from the single input RS and blurred image by employing the single-view joint RS correction and deblurring CNN;
• wherein single-view joint RS correction and deblurring is performed in a single forward pass.

A system for jointly removing rolling shutter (RS) distortions and blur artifacts in a single input RS and blurred image, the system comprising:

• a memory;
• and
• one or more processors in communication with the memory configured to:
• generate a plurality of RS blurred images from a camera;
• synthesize RS blurred images from a set of OS sharp images, corresponding GS sharp depth maps, and synthesized RS camera motions by employing a structure-and-motion-aware RS distortion and blur rendering module to generate training data to train a single-view joint RS correction and deblurring convolutional neural network (CNN);
• and
• predict an RS rectified and deblurred image from the single input RS and blurred image by employing the single-view joint RS correction and deblurring CNN;
• wherein single-view joint RS correction and deblurring is performed in a single forward pass.