Taken-image signal-distortion compensation method, taken-image signal-distortion compensation apparatus, image taking method and image-taking apparatus

A method of compensating a signal of an image of one screen for distortions in the horizontal and/or vertical directions of a reference screen, said method comprising the steps of:

• dividing one screen area of said image into a plurality of picture divisions;
• detecting a movement vector of said image for each particular one of said picture divisions from information on a difference in image between said particular picture division on an observed screen serving as said reference screen and said particular picture division on an original screen leading ahead of said reference screen;
• and
• compensating said image for a distortion for each of said picture divisions on the basis of said movement vector detected for each of said picture divisions,
• wherein said step of detecting a movement vector of said image for each particular one of said picture divisions includes the sub-steps of
• setting at least one target block having a predetermined size equal to the size of a plurality of pixels at predetermined locations in each of said picture divisions on said original screen,
• setting a plurality of reference blocks each having a size equal to said predetermined size of said target block in a search range set in said reference screen for each of said target blocks,
• detecting a reference block having the strongest correlation with said specific target block from said plurality of reference blocks, and
• detecting said movement vector representing the magnitude and direction of a shift over said reference screen from a position corresponding to said specific target block to the position of said detected reference block by carrying out
• a difference absolute value sum computation process for each specific one of said reference blocks as a process of finding a sum of the absolute values of differences in pixel value between all pixels in said specific reference block and all pixels at corresponding positions on said specific target block,
• a contracted reference-vector setting process of setting reference vectors each representing the magnitude and direction of a shift over said reference screen from a position corresponding to said specific target block to the position of one of said reference blocks and contracting said reference vectors at a predetermined contraction factor to result in contracted reference vectors,
• a sum-of-absolute-difference table creation process of creating a shrunk sum-of-absolute-difference table including table elements, the number of which is determined on the basis of the number of said contracted reference vectors and said contraction factor, and allocating each of said table elements as a location to be used for storing a value computed from sums each found for one of said reference blocks as said sum of the absolute values of said differences in pixel values, and
• a movement-vector computation process of computing said movement vector for each of said picture divisions as a movement vector between said reference screen and said original screen by making use of at least a contracted reference vector corresponding to the smallest one among values each stored in said shrunk sum-of-absolute-difference table as said sum of the absolute values of said differences in pixel values,
• whereby said sum-of-absolute-difference table creation process is carried out by execution of
• a neighborhood reference vector detection sub-process of determining a plurality of neighborhood reference vectors each having a vector quantity close to the vector quantity of a corresponding one of said contracted reference vectors obtained in said contracted reference-vector setting process,
• a sum-component computation sub-process of computing a plurality of sum components each associated with one of neighborhood reference vectors determined in said neighborhood reference vector detection sub-process as said neighborhood reference vectors corresponding to a specific one of said contracted reference vectors from a sum computed in said difference absolute value sum computation process for a reference block, from which said specific contracted reference vector is contracted, as said sum of the absolute values of said differences in pixel values, and
• a component-total computation sub-process of computing a total of sum components, which have been computed in said sum-component computation sub-process as said sum components each associated with a specific one of said neighborhood reference vectors, by cumulatively adding each of said sum components to a cumulative total obtained so far for said specific neighborhood reference vector.

An image taking method for compensating an imager-created image for a distortion generated by a positional change caused by a hand movement as a positional change of an image pickup device and for recording information of said compensated image onto a recording medium, said image taking method comprising the steps of:

• dividing one screen area of said image into a plurality of picture divisions;
• detecting a movement vector of an image for each particular one of said picture divisions from information on a difference in image between said particular picture division on one of two screens and said particular picture division on the other of said two screens;
• detecting a movement vector of said imager-created image for each particular one of said picture divisions from information on a difference in image between said particular picture division on an observed screen and said particular picture division on an original screen leading ahead of said reference screen;
• finding a difference in said detected movement vector between any specific picture division and a picture division adjacent to said specific picture division in order to detect a velocity of said positional change for said specific picture division;
• compensating said imager-created image for a distortion for each of said picture divisions on the basis of a velocity detected for each of said picture divisions as said velocity of said positional change;
• and
• recording image information of said compensated imager-created image onto a recording medium;
• wherein said step of detecting a movement vector of said image for each particular one of said picture divisions comprises the sub-steps of:
• making use of images, which are an observed screen serving as a reference screen and an original screen leading ahead of said reference screen for detecting said movement vector;
• setting at least one target block having a predetermined size equal to the size of a plurality of pixels at predetermined locations in each of said picture divisions on said original screen;
• setting a plurality of reference blocks each having a size equal to said target block in a search range set in said reference screen for each of said target blocks;
• detecting a reference block having the strongest correlation with said specific target block from said plurality of reference blocks;
• detecting said movement vector representing the magnitude and direction of a shift over said reference screen from a position corresponding to said specific target block to the position of said detected reference block;
• wherein said movement vector of said image is detected for each of said picture divisions by carrying out:
• a difference absolute value sum computation process for each specific one of said reference blocks as a process of finding a sum of the absolute values of differences in pixel value between all pixels in said specific reference block and all pixels at corresponding positions on said specific target block;
• a contracted reference-vector setting process of setting reference vectors each representing the magnitude and direction of a shift over said reference screen from a position corresponding to said specific target block to the position of one of said reference blocks and contracting said reference vectors at a predetermined contraction factor to result in contracted reference vectors;
• a sum-of-absolute-difference table creation process of creating a shrunk sum-of-absolute-difference table including table elements, the number of which is determined on the basis of the number of said contracted reference vectors and said contraction factor, and allocating each of said table elements as a location to be used for storing a value computed from sums each found for one of said reference blocks as said sum of the absolute values of said differences in pixel values;
• and
• a movement-vector computation process of computing said movement vector for each of said picture divisions as a movement vector between said reference screen and said original screen by making use of at least a contracted reference vector corresponding to the smallest one among values each stored in said shrunk sum-of-absolute-difference table as said sum of the absolute values of said differences in pixel values, and
• said sum-of-absolute-difference table creation process is carried out by execution of
• a neighborhood reference vector detection sub-process of determining a plurality of neighborhood reference vectors each having a vector quantity close to the vector quantity of a corresponding one of said contracted reference vectors obtained in said contracted reference-vector setting process,
• a sum-component computation sub-process of computing a plurality of sum components each associated with one of neighborhood reference vectors determined in said neighborhood reference vector detection sub-process as said neighborhood reference vectors corresponding to a specific one of said contracted reference vectors from a sum computed in said difference absolute value sum computation process for a reference block, from which said specific contracted reference vector is contracted, as said sum of the absolute values of said differences in pixel values, and
• a component-total computation sub-process of computing a total of sum components, which have been computed in said sum-component computation sub-process as said sum components each associated with a specific one of said neighborhood reference vectors, by cumulatively adding each of said sum components to a cumulative total obtained so far for said specific neighborhood reference vector.

An apparatus for compensating a signal of an image of one screen for distortions in the horizontal and/or vertical directions of said image, comprising:

• a movement-vector detection unit configured to detect a movement vector of said image for each particular one of picture divisions obtained as a result of dividing one screen area of an image of a screen from information on a difference in image between said particular picture division on an observed screen serving as a reference screen and said particular picture division on an original screen leading ahead of said reference screen;
• a velocity detection unit configured to detect the velocity of a change in image distortion for each specific one of said picture divisions by finding a difference between said movement vector detected by said movement-vector detection unit for said specific picture division and said movement vector detected by said movement-vector detection unit for a picture division adjacent to said specific picture division;
• and
• a distortion compensation unit configured to compensate said image for a distortion for each specific one of said picture divisions on the basis of a velocity detected by said velocity detection unit as said velocity of a change in image distortion for said specific picture division;
• wherein said distortion of said image is a distortion of an image created in an image pickup device due to a positional change made by said image pickup device at a photographing time in a horizontal and/or vertical directions of a photographing object;
• wherein each of said picture divisions includes a plurality of lines,
• data of said image is read out from said image pickup device in line units each corresponding to any one of said lines, and
• said distortion compensation unit further comprises:
• a displacement quantity integration unit configured to carry out an integration process with respect to time in each particular one of said picture divisions on a velocity detected for said particular picture division as said velocity of a change in image distortion to time up to any specific one of said line units to give a displacement quantity;
• and
• an image compensation unit configured to compensate said image for a distortion by making use of said displacement quantity found by said displacement quantity integration unit as a displacement quantity at the position of said specific line unit;
• wherein said displacement quantity integration unit employed in said distortion compensation unit further comprises:
• a velocity splitting unit configured to split said velocity of a change in image distortion into a horizontal-direction velocity component and a vertical-direction velocity component;
• a horizontal-direction displacement computation unit configured to integrate said horizontal-direction velocity component obtained for any particular one of said picture divisions with respect to time up to any specific one of said line units in each particular one of said picture divisions to give a horizontal-direction displacement quantity to be used as a horizontal-direction displacement quantity at the position of said specific line unit;
• and
• a vertical-direction displacement computation unit configured to integrate said vertical-direction velocity component obtained for any particular one of said picture divisions with respect to time up to any specific one of said line units in each particular one of said picture divisions to give a vertical-direction displacement quantity to be used as a vertical-direction displacement quantity at the position of said specific line unit;
• and
• said image compensation unit employed in said distortion compensation unit further comprises:
• a horizontal-direction compensation unit configured to compensate said image for a horizontal-direction distortion by making use of a horizontal-direction displacement quantity found by said horizontal-direction displacement computation unit as said horizontal-direction displacement quantity at the position of each of said line units in each of said picture divisions;
• and
• a vertical-direction compensation unit configured to compensate said image for a vertical-direction distortion by making use of a vertical-direction displacement quantity found by said vertical-direction displacement computation unit as said vertical-direction displacement quantity at the position of each of said line units in each of said picture divisions.

An apparatus for compensating a signal of an image of one screen for distortions in the horizontal and/or vertical directions of said image, comprising:

• a movement-vector detection unit configured to detect a movement vector of said image for each particular one of picture divisions obtained as a result of dividing one screen area of an image of a screen from information on a difference in image between said particular picture division on an observed screen serving as a reference screen and said particular picture division on an original screen leading ahead of said reference screen;
• a velocity detection unit configured to detect the velocity of a change in image distortion for each specific one of said picture divisions by finding a difference between said movement vector detected by said movement-vector detection unit for said specific picture division and said movement vector detected by said movement-vector detection unit for a picture division adjacent to said specific picture division;
• and
• a distortion compensation unit configured to compensate said image for a distortion for each specific one of said picture divisions on the basis of a velocity detected by said velocity detection unit as said velocity of a change in image distortion for said specific picture division;
• wherein said movement-vector detection unit is a unit for finding a movement vector between an observed screen serving as a reference screen and an original screen leading ahead of said reference screen for each of said picture divisions;
• at least one target block having a predetermined size equal to the size of a plurality of pixels are set at predetermined locations in each of said picture divisions on said original screen;
• a plurality of reference blocks each having a size equal to said predetermined size of said target block are set in a search range set in said reference screen for each of said target blocks;
• said search range set for each specific one of said target blocks is searched for a reference block having the strongest correlation with said specific target block;
• and
• said movement vector representing the magnitude and direction of a shift over said reference screen from a position corresponding to said specific target block to the position of said detected reference block is detected;
• wherein said movement-vector detection unit comprises:
• a difference absolute value sum computation unit configured to find a sum for each specific one of said reference blocks as a sum of the absolute values of differences in pixel value between all pixels in said specific reference block and all pixels at corresponding positions on said specific target block;
• a contracted reference-vector setting unit configured to set reference vectors each representing the magnitude and direction of a shift over said reference screen from a position corresponding to said specific target block to the position of one of said reference blocks and contract said reference vectors at a predetermined contraction factor to result in contracted reference vectors;
• a sum-of-absolute-difference table creation unit configured to create a shrunk sum-of-absolute-difference table including table elements, the number of which is determined on the basis of the number of said contracted reference vectors and said contraction factor, and allocate each of said table elements as a location to be used for storing a value computed from sums each found for one of said reference blocks as said sum of the absolute values of said differences in pixel values;
• and
• a movement-vector computation unit configured to compute said movement vector for each of said picture divisions as a movement vector between said reference screen and said original screen by making use of at least a contracted reference vector corresponding to the smallest one among values each stored in said shrunk sum-of-absolute-difference table as said sum of the absolute values of said differences in pixel values;
• and
• said sum-of-absolute-difference table creation unit includes,
• a neighborhood reference vector detection unit configured to determine a plurality of neighborhood reference vectors each having a vector quantity close to the vector quantity of a corresponding one of said contracted reference vectors obtained by said contracted reference-vector setting unit,
• a sum-component computation unit configured to compute a plurality of sum components each associated with one of neighborhood reference vectors determined by said neighborhood reference vector detection unit as said neighborhood reference vectors corresponding to a specific one of said contracted reference vectors from a sum computed by said difference absolute value sum computation unit for a reference block pointed to by a reference vector, from which said specific contracted reference vector is contracted, as said sum of the absolute values of said differences in pixel values, and
• a component-total computation unit configured to compute a total of sum components, which have been computed by said sum-component computation unit as said sum components each associated with a specific one of said neighborhood reference vectors, by cumulatively adding each of said sum components to a cumulative total obtained so far for said specific neighborhood reference vector.

An apparatus for compensating a signal of an image of one screen for distortions in the horizontal and/or vertical directions of said image, comprising:

• a movement-vector detection unit configured to detect a movement vector of said image for each particular one of picture divisions obtained as a result of dividing one screen area of an image of a screen from information on a difference in image between said particular picture division on an observed screen serving as a reference screen and said particular picture division on an original screen leading ahead of said reference screen;
• a velocity detection unit configured to detect the velocity of a change in image distortion for each specific one of said picture divisions by finding a difference between said movement vector detected by said movement-vector detection unit for said specific picture division and said movement vector detected by said movement-vector detection unit for a picture division adjacent to said specific picture division;
• and
• a distortion compensation unit configured to compensate said image for a distortion for each specific one of said picture divisions on the basis of a velocity detected by said velocity detection unit as said velocity of a change in image distortion for said specific picture division;
• wherein said distortion of said image is a distortion of an image created in an image pickup device due to a positional change made by said image pickup device at a photographing time in a horizontal and/or vertical directions of a photographing object;
• wherein each of said picture divisions includes a plurality of lines,
• data of said image is read out from said image pickup device in line units each corresponding to any one of said lines, and
• said distortion compensation unit further comprises:
• a displacement quantity integration unit configured to carry out an integration process with respect to time in each particular one of said picture divisions on a velocity detected for said particular picture division as said velocity of a change in image distortion to time up to any specific one of said line units to give a displacement quantity;
• and
• an image compensation unit configured to compensate said image for a distortion by making use of said displacement quantity found by said displacement quantity integration unit as a displacement quantity at the position of said specific line unit;
• wherein said displacement quantity integration unit employed in said distortion compensation unit further comprises:
• a velocity splitting unit configured to split said velocity of a change in image distortion into a horizontal-direction velocity component and a vertical-direction velocity component;
• a horizontal-direction displacement computation unit configured to integrate said horizontal-direction velocity component obtained for any particular one of said picture divisions with respect to time up to any specific one of said line units in each particular one of said picture divisions to give a horizontal-direction displacement quantity to be used as a horizontal-direction displacement quantity at the position of said specific line unit;
• and
• a vertical-direction displacement computation unit configured to integrate said vertical-direction velocity component obtained for any particular one of said picture divisions with respect to time up to any specific one of said line units in each particular one of said picture divisions to give a vertical-direction displacement quantity to be used as a vertical-direction displacement quantity at the position of said specific line unit;
• and
• said image compensation unit employed in said distortion compensation unit further comprises:
• a horizontal-direction compensation unit configured to compensate said image for a horizontal-direction distortion by making use of a horizontal-direction displacement quantity found by said horizontal-direction displacement computation unit as said horizontal-direction displacement quantity at the position of each of said line units in each of said picture divisions;
• and
• a vertical-direction compensation unit configured to compensate said image for a vertical-direction distortion by making use of a vertical-direction displacement quantity found by said vertical-direction displacement computation unit as said vertical-direction displacement quantity at the position of each of said line units in each of said picture divisions;
• wherein said movement-vector detection unit:
• further divides each of said picture divisions into a plurality of picture sub-divisions;
• sets said target block in each of said picture sub-divisions;
• detects said movement vector for each of said target block;
• and
• detects a movement vector for any specific one of said picture divisions from a plurality of said movement vectors each detected for one of said target blocks set in said specific picture division.

An apparatus for compensating an image, said apparatus comprising:

• a movement-vector detection unit configured to detect a movement vector of said image for each particular one of picture divisions obtained as a result of dividing one screen area of an image of a screen from information on a difference in image between said particular picture division on an observed screen serving as a reference screen and said particular picture division on an original screen leading ahead of said reference screen;
• and
• an image-distortion compensation unit configured to compensate said image for distortions in the horizontal and/or vertical directions of said observed screen for each of said picture divisions on the basis of said movement vector detected for each of said picture divisions, wherein,
• said movement-vector detection unit has
• a target-block setting unit configured to set at least one target block having a predetermined size including a plurality of pixels at predetermined locations in each of said picture divisions on said original screen,
• a reference-block setting unit configured to set a plurality of reference blocks each having a size equal to said predetermined size of said target block in a search range set in said reference screen for each of said target blocks,
• a reference-block detecting unit configured to detect a reference block having the strongest correlation with said specific target block from said plurality of reference blocks, and
• a movement-vector detection unit configured to detect said movement vector representing the magnitude and direction of a shift over said reference screen from a position corresponding to said specific target block to the position of said detected reference block,
• said movement-vector detection unit includes
• a difference absolute value sum computation section configured to find a sum of the absolute values of differences in pixel value between all pixels in each of said reference blocks and all pixels at corresponding positions on said specific target block for each of said reference blocks,
• a contracted reference-vector setting section configured to set reference vectors each representing the magnitude and direction of a shift over said reference screen from a position corresponding to said specific target block to the position of one of said reference blocks and contract each of said reference vectors at a predetermined contraction factor to result in contracted reference vectors,
• a sum-of-absolute-difference table creation section configured to create a shrunk sum-of-absolute-difference table including table elements, the number of which is determined on the basis of the number of said contracted reference vectors and said contraction factor, and allocate each of said table elements as a location to be used for storing a value computed from sums each found for one of said reference blocks as said sum of the absolute values of said differences in pixel values, and
• a movement-vector computation section configured to compute said movement vector for each of said picture divisions as a movement vector between said reference screen and said original screen by making use of at least a contracted reference vector corresponding to the smallest one among values each stored in said shrunk sum-of-absolute-difference table as said sum of the absolute values of said differences in pixel values, and
• said sum-of-absolute-difference table creation section includes
• a neighborhood reference vector detection sub-section configured to determine a plurality of neighborhood reference vectors each having a vector quantity close to the vector quantity of a corresponding one of said contracted reference vectors obtained by said contracted reference-vector setting section,
• a sum-component computation sub-section configured to compute a plurality of sum components each associated with one of neighborhood reference vectors determined by said neighborhood reference vector detection sub-section as said neighborhood reference vectors corresponding to a specific one of said contracted reference vectors from a sum computed by said difference absolute value sum computation section for a reference block pointed to by a reference vector, from which said specific contracted reference vector is contracted, as said sum of the absolute values of said differences in pixel values, and
• a component-total computation sub-section configured to compute a total of sum components, which have been computed by said sum-component computation sub-section as said sum components each associated with a specific one of said neighborhood reference vectors, by cumulatively adding each of said sum components to a cumulative total obtained so far for said specific neighborhood reference vector.