US 2019/0158768 A1Published application

Systems and methods for rolling shutter compensation using iterative process

AssigneeGoPro, Inc.

InventorAdrien Fontvielle

Life of this patent

Estimating the term…

Loading patent…

Abstract

Image captured with an image capture device with a rolling shutter may be deformed due to changes in imaging sensor orientation during image capture. Image deformities may occur due to rolling shutter that exposes rows of pixels to light at slightly different times during image capture. Deformities such as wobble, for example, and/or other deformities may be corrected by constructing an output image. The output image may be constructed by determining corresponding pixels within the input image. The location of the input pixel may be determined by performing one or more fixed point iterations to identify one or more input pixels within the input image. A value of the output pixel within the output image may be determined based on a value of a corresponding pixel within the input image.

Claims20

Dependent claims

A system for correcting digital image deformities, the system comprising:

one or more physical processors configured by machine readable instructions to:
obtain an input image defined by an input pixel array, the input pixel array captured by an imaging sensor, the input pixel array including input pixels characterized by input pixel positions within the input pixel array and input pixel values;
obtain acquisition times specifying time of capture of sets of input pixels within the input pixel array;
obtain orientation information specifying imaging sensor orientations at the acquisition times of the sets of input pixels within the input pixel array;
determine an output image defined by an output pixel array, the output pixel array determined based on the input pixel array of the input image, the output pixel array including output pixels characterized by output pixel positions within the output pixel array and output pixel values, wherein determination of a given output pixel includes:
(a) determining an acquisition time and an orientation information of the given output pixel within the output pixel array;
(b) obtaining identification of an input pixel based on the acquisition time and the orientation information of the given output pixel, the input pixel having a pixel position within the input pixel array;
(c) obtaining an input acquisition time of the input pixel based on the pixel position within the input pixel array;
(d) determining input orientation information of the input pixel based on the input acquisition time;
(e) obtaining identification of a subsequent input pixel based on the input orientation information;
(f) iterating over operations (c)-(e) for one or more other pixels of the input pixel array;
and
(g) obtaining identification of a pixel within the input pixel array as an estimated input pixel based on differences in the input orientation information;
and
determining the output pixel values for the given output pixel based on the input pixel values for the estimated input pixel.

12 more elements

A method of correcting digital image deformities, the method comprising:

obtaining an input image defined by an input pixel array, the input pixel array captured by an imaging sensor, the input pixel array including input pixels characterized by pixel positions within the input pixel array and input pixel values;
obtaining acquisition times specifying time of capture of sets of input pixels within the input pixel array;
obtaining orientation information specifying imaging sensor orientations at the acquisition times of the sets of input pixels within the input pixel array;
determining an output image defined by an output pixel array, the output pixel array determined based on the input pixel array of the input image, the output pixel array including output pixels characterized by output pixel positions within the output pixel array and output pixel values, wherein determination of a given output pixel includes:
(a) determining an acquisition time and an orientation information of the given output pixel within the output pixel array;
(b) obtaining identification of an input pixel based on the acquisition time and the orientation information of the given output pixel, the input pixel having a pixel position within the input pixel array;
(c) obtaining an input acquisition time of the input pixel based on the pixel position within the input pixel array;
(d) determining input orientation information of the input pixel based on the input acquisition time;
(e) obtaining identification of a subsequent input pixel based on the input orientation information;
(f) iterating over operations (c)-(e) for one or more other pixels of the input pixel array;
and
(g) obtaining identification of a pixel within the input pixel array as an estimated input pixel based on differences in the input orientation information;
and
determining the output pixel values for the given output pixel based on the input pixel values for the estimated input pixel.

11 more elements

Dependencies are inferred from the claim text.

Facts

Application: 16/257,290Filed 25 Jan 2019
Priority: 22 Feb 2017
Published: 23 May 2019
Term ends: Not estimatedThe source records don’t support an estimate.

Classification6 codes

G06T 3/60
Geometric image transformations in the plane of the image: rotation of whole images or parts thereof
H04N 23/6812
Motion detection: based on additional sensors, e.g. acceleration sensors
H04N 23/683
Vibration or motion blur correction: performed by a processor, e.g. controlling the readout of an image memory
H04N 25/531
Control of the integration time: by controlling rolling shutters in CMOS SSIS

Family

Application15/439,257Continuation · filed 22 Feb 2017
Application16/257,290Continuation
US 10,194,101 B1Related patent
US 10,412,328 B2Related patent
US 2019/0158768 A1This patent

From the US filing history. Foreign family members aren’t listed.