US 8,264,551 B2Granted

Methods for correcting distortions of image-taking video signals and apparatus for correcting distortions of image-taking video signals

AssigneeSony Corporation

InventorsTohru Kurata, Manabu Kubo, Kimitaka Wada, Toshiyuki Kaimi

Life of this patent

Estimating the term…

Loading patent…

Abstract

A method and an apparatus for correcting distortions of image-taking video signals are provided. The method and an apparatus adopting the method are capable of reducing distortion generated in a taken image by correction of inter-frame and intra-frame hand movements in an image-taking apparatus employing an image-taking device of an X-Y address type. To be more specific, this method is capable of correcting distortion observed in a taken image as distortion caused by a positional change, which occurs in the horizontal and/or vertical directions of the taken image at a photographing time as a positional change of the image-taking device. In accordance with the method, in a frame period of a taken image, an inter-frame correction quantity for a positional change of the device is detected and, on the basis of the detected inter-frame correction quantity, the positional change is corrected to eliminate a displacement observed when the frame changes to the next one as a displacement of the taken image. An intra-frame correction quantity for a positional change of the image-taking device is computed for each of many locations on the screen of the taken image and, on the basis of the computed intra-frame correction quantity, distortions caused by positional changes on the screen of the taken image as distortions of the taken image are corrected.

Claims16

Dependent claims

A method for correcting distortions contained in a signal representing a taken image of a moving picture as distortions caused by a positional change, which occurs in at least one of a horizontal and a vertical direction of said taken image at a time to photograph said moving picture as a positional change of an image-taking device, said method comprising the steps of:

correcting an inter-frame, which detects an inter-frame correction quantity for said positional change of said image-taking device in a frame period of said taken image and corrects said positional change by elimination of a displacement occurring between a frame and a next frame as a displacement of said taken image on the basis of said detected inter-frame correction quantity;
and
correcting an intra-frame, which computes an intra-frame correction quantity for said positional change of said image-taking device at each of a plurality of locations on one screen of said taken image and corrects a distortion caused by said positional change on said screen of said taken image as a distortion of said taken image on the basis of said computed intra-frame correction quantity, wherein said correcting said intra-frame comprises the steps of:
(a) dividing one screen segment of said taken image into a plurality of image sub-segments,
(b) detecting a rate of said positional change for each of said image sub-segments, wherein each sub-segment is arranged to include a plurality of pixels, and a first rate of a first sub-segment defined by a first hand-movement velocity vector is different than a second rate of a second sub-segment defined by a second hand-movement velocity vector,
(c) integrating said detected rate including a horizontal-direction component and a vertical-direction component for each of said image sub-segments over the period of said image sub-segment in order to find a time-integration value to be used as said intra-frame correction quantity for a member subjected to a distortion correction process as a member of said taken image,
(d) suspending said integrating of the vertical-direction component in response to a rate of progress of said integrating of the vertical-direction component being greater than a rate of progress of said integrating of the horizontal-direction component, and
(e) correcting a distortion caused by said positional change on one screen of said taken image as a distortion in said taken image using said time-integration value to correct a distortion of said taken image.

5 more elements

(a) dividing said positional change into a horizontal-direction component and a vertical-direction component;
(b) correcting an inter-frame, which:
(i) detects an inter-frame horizontal correction quantity for said horizontal-direction component of said positional change of said image-taking device for each frame period of said taken image;
(ii) detects an inter-frame vertical correction quantity for said vertical-direction component of said positional change of said image-taking device for each frame period of said taken image;
and
(iii) computes next-inter-frame taken-image head coordinates on the basis of said detected inter-frame horizontal correction quantity and said detected inter-frame vertical correction quantity;
(c) dividing one screen segment of said taken image into a plurality of image sub-segments;
(d) detecting a rate of said positional change for each of said image sub-segments, wherein each sub-segment is arranged to include a plurality of pixels, and a first rate of a first sub-segment defined by a first hand-movement velocity vector is different than a second rate of a second sub-segment defined by a second hand-movement velocity vector, wherein the rate of said positional change is divided into a horizontal-direction component and a vertical-direction component;
(e) correcting an intra-frame horizontally, which:
(i) computes an intra-frame horizontal correction quantity for said horizontal-direction component of said positional change of said image-taking device at each of a plurality of positions on one screen of said taken image, wherein a horizontal-direction displacement for each line in each of said image sub-segments is found by carrying out a time-integration process progressing along the time axis in line units on said horizontal-direction component of said rate of said positional change and used for correcting a horizontal-direction distortion of said taken image;
(ii) adds said computed intra-frame horizontal correction quantity to a horizontal coordinate of said head coordinates computed at said inter-frame correction step to find a horizontal correction quantity;
and
(iii) corrects a horizontal-direction distortion in a frame of said taken image by using said found horizontal correction quantity using said time-integration value to correct a distortion of said taken image;
and
(f) correcting an intra-frame vertically, which:
(i) computes an intra-frame vertical correction quantity for said vertical-direction component of said positional change of said image-taking device at each of a plurality of positions on one screen of said taken image, wherein a vertical-direction displacement for each line in each of said image sub-segments is found by carrying out a time-integration process progressing along the time axis in line units on said vertical-direction component of said rate of said positional change and used for correcting a vertical-direction distortion of said taken image;
(ii) suspends said time-integration process of the vertical-direction component in response to a rate of progress of said time-integration process of the vertical-direction component being greater than a rate of progress of said time-integration process of the horizontal-direction component;
(ii) adds said computed intra-frame vertical correction quantity to a vertical coordinate of said head coordinates computed at said inter-frame correction step to find a vertical correction quantity;
and
(iii) corrects a vertical-direction distortion in a frame of said taken image by using said found vertical correction quantity using said time-integration value to correct a distortion of said taken image.

17 more elements

An apparatus for correcting distortions contained in a signal representing a taken image of a moving picture as distortions caused by a positional change, which occurs in at least one of a horizontal and a vertical direction of said taken image at a time to photograph said moving picture as a positional change of an image-taking device, said apparatus comprising:

inter-frame correction means for detecting an inter-frame correction quantity for said positional change of said image-taking device in a frame period of said taken image and correcting said positional change by elimination of a displacement occurring between a frame and the next frame as a displacement of said taken image on the basis of said detected inter-frame correction quantity;
and
intra-frame correction means for computing an intra-frame correction quantity for said positional change of said image-taking device at each of a plurality of locations on one screen of said taken image and correcting a distortion caused by said positional change on said screen of said taken image as a distortion of said taken image on the basis of said computed intra-frame correction quantity,
wherein said intra-frame correction means includes:
velocity detection means for detecting a rate of said positional change for each of a plurality of image sub-segments obtained as a result of dividing one screen segment of said taken image into said plurality of image sub-segments, wherein each sub-segment is arranged to include a plurality of pixels, and a first rate of a first sub-segment defined by a first hand-movement velocity vector is different than a second rate of a second sub-segment defined by a second hand-movement velocity vector,
distortion correction means for computing said intra-frame correction quantity for a member subjected to a distortion correction process as a member of said taken image on the basis of said rate detected for each of said image sub-segments as said rate of said positional change in each of said image sub-segments and correcting a distortion caused by said positional change on one screen of said taken image as a distortion in said taken image, said distortion correction means further comprises:
displacement integration means for integrating said detected rate includes a horizontal-direction component and a vertical-direction component for each of said image sub-segments over the period of said image sub-segment in order to find a time-integration value to be used as said intra-frame correction quantity for a member subjected to a distortion correction process as a member of said taken image, wherein said integrating of the vertical-direction component is suspended in response to a rate of progress of said integrating of the vertical-direction component being greater than a rate of progress of said integrating of the horizontal-direction component, and
distortion correction means for correcting a distortion of said taken image by using said time-integration value.

5 more elements

(a) inter-frame correction means for:
(i) detecting an inter-frame horizontal correction quantity for said horizontal-direction component of said positional change of said image-taking device for each frame period of said taken image;
(ii) detecting an inter-frame vertical correction quantity for said vertical-direction component of said positional change of said image-taking device for each frame period of said taken image;
and
(iii) computing next-inter-frame taken-image head coordinates on the basis of said detected inter-frame horizontal correction quantity and said detected inter-frame vertical correction quantity;
(b) intra-frame horizontal correction means for:
(i) computing an intra-frame horizontal correction quantity for said horizontal-direction component of said positional change of said image-taking device at each of a plurality of positions on one screen of said taken image;
(ii) adding said computed intra-frame horizontal correction quantity to a horizontal coordinate of said head coordinates computed by said inter-frame correction means to find a horizontal correction quantity;
and
(iii) correcting a horizontal-direction distortion in a frame of said taken image by using said found horizontal correction quantity;
and
(c) intra-frame vertical correction means for:
(i) computing an intra-frame vertical correction quantity for said vertical-direction component of said positional change of said image-taking device at each of a plurality of positions on one screen of said taken image;
(ii) adding said computed intra-frame vertical correction quantity to a vertical coordinate of said head coordinates computed by said inter-frame correction means to find a vertical correction quantity;
and
(iii) correcting a vertical-direction distortion in a frame of said taken image by using said found vertical correction quantity;
and
(d) velocity detection means for detecting a rate of said positional change for each of a plurality of image sub-segments obtained as a result of dividing one screen segment of said taken image into said plurality of image sub-segments, wherein each sub-segment is arranged to include a plurality of pixels, and a first rate of a first sub-segment defined by a first hand-movement velocity vector is different than a second rate of a second sub-segment defined by a second hand-movement velocity vector,
wherein said intra-frame horizontal correction means and said intra-frame vertical correction means divide the rate of said positional change into a horizontal-direction component and a vertical-direction component,
wherein said intra-frame horizontal correction means finds a horizontal-direction displacement for each line in each of said image sub-segments by carrying out a time-integration process progressing along the time axis in line units on said horizontal-direction component of said rate of said positional change and uses said horizontal-direction displacement for correcting a horizontal-direction distortion of said taken image,
wherein said intra-frame vertical correction means finds a vertical-direction displacement for each line in each of said image sub-segments by carrying out a time-integration process progressing along the time axis in line units on said vertical-direction component of said rate of said positional change and uses said vertical-direction displacement for correcting a vertical-direction distortion of said taken image,
wherein said intra-frame vertical correction means suspends said time-integration process of the vertical-direction component in response to a rate of progress of said time-integration process of the vertical-direction component being greater than a rate of progress of said time-integration process of the horizontal-direction component.

19 more elements

Dependencies are inferred from the claim text.

Facts

Application: 11/317,845Filed 22 Dec 2005
Priority: JP P2004-38068328 Dec 2004
Published: US 2006/0140603 A129 Jun 2006
Granted: 11 Sep 2012
Term ends: Not estimatedThe source records don’t support an estimate.

Classification4 codes

H04N 23/68
Control of cameras or camera modules: for stable pick-up of the scene, e.g. compensating for camera body vibrations
H04N 23/683
Vibration or motion blur correction: performed by a processor, e.g. controlling the readout of an image memory
H04N 23/6842
Performed by controlling the image sensor readout, e.g. by controlling the integration time: by controlling the scanning position, e.g. windowing
H04N
Pictorial communication