Rolling shutter reduction based on motion sensors

A method for reducing rolling shutter distortion in an input frame, comprising:

• determining motion of each input row of the input frame;
• finding a motion difference between the motion of a base row of the input frame and each of the input rows;
• and
• generating a plurality of transformed rows according to a plurality of corresponding perspective transforms, wherein the perspective transforms correct for shutter distortion associated with input rows based on the motion difference.

A non-transitory computer-readable storage medium storing computer-executable instructions that, when executed, cause a processor to execute a method comprising:

• determining motion of each input row of an input frame;
• finding a motion difference between the motion of a base row of the input frame and each of the input rows;
• and
• generating a plurality of transformed rows according to a plurality of corresponding perspective transforms, wherein the perspective transforms correct for shutter distortion associated with input rows based on the motion difference.

An apparatus comprising:

• a image sensor to capture and output a sequence of input rows of an input frame;
• a motion sensor to output motion data corresponding to motion of the apparatus at approximately the time the image sensor captures the sequence of input rows;
• a memory to store a corrected frame;
• and
• a controller configured to:
• estimate motion of each input row based on the motion data;
• find a motion difference for each row between the motion of the row and a base row of the input frame;
• generate a perspective transform for each row based on the motion difference of the corresponding row;
• apply the corresponding generated perspective transform to each row to generate a corrected row;
• assemble the corrected rows into the corrected frame;
• and
• store the corrected frame in the memory.

A chip comprising:

• a video device driver configured to receive a captured frame including a plurality of captured rows;
• a motion sensor driver configured to receive motion data associated with the captured rows;
• and
• a rolling shutter distortion processor configured to:
• determine motion of each captured row of the captured frame;
• find a motion difference between the motion of a base row of the captured frame and each of the captured rows;
• and
• generate a plurality of transformed rows according to a plurality of corresponding perspective transforms, wherein the perspective transforms correct for shutter distortion associated with input rows based on the motion difference.

A non-transitory computer-readable storage medium storing computer-executable instructions that, when executed, cause a processor to perform a method comprising:

• estimating motion of a base row of the frame based on motion sensor data representing motion of image sensor during capture;
• for a plurality of other rows of the frame, estimating a motion difference between motion of each other row and the base row based on the motion sensor data;
• generating a perspective transform for each other row based on the estimated motion difference;
• and
• applying the generated perspective transforms to image data of the rows to generate corrected frame data.

An apparatus comprising:

• a image sensor to capture and output a sequence of rows of a frame of image data;
• a motion sensor to output motion data corresponding to motion of the apparatus at approximately the time the image sensor captures the sequence of rows;
• a memory to store a corrected frame;
• and
• a controller configured to:
• estimate motion of a base row of the frame based on the motion data representing motion of the apparatus during capture;
• for a plurality of other rows of the frame, estimate a motion difference between motion of each other row and the base row based on the motion data;
• generate a perspective transform for each other row based on the estimated motion difference;
• and
• apply the generated perspective transforms to image data of the rows to generate corrected frame data.

A chip comprising:

• a video device driver configured to receive a captured frame including a plurality of captured rows;
• a motion sensor driver configured to receive motion data associated with the captured rows;
• and
• a rolling shutter distortion processor configured to:
• estimate motion of a base row of the frame based on the motion data;
• for a plurality of other rows of the frame, estimate a motion difference between motion of each other row and the base row based on the motion data;
• generate a perspective transform for each other row based on the estimated motion difference;
• and
• apply the generated perspective transforms to image data of the rows to generate corrected frame data.