Image processing device, image processing method, and program for image processing for correcting displacement between pictures obtained by temporally-continuous capturing

An image processing method for correcting displacement between pictures obtained by temporally-continuous capturing, the image processing method comprising:

• estimating a motion amount indicating a displacement amount of a second picture with respect to a first picture using feature points extracted from each of the first picture and the second picture, the second picture being captured temporally after the first picture;
• determining, using the feature points, whether or not correction is made using the motion amount estimated by the motion amount estimating unit;
• and
• correcting the displacement of the second picture with respect to the first picture using the motion amount so as to correct the displacement between the pictures, in the case where the determining involves determining that the correction is made using the motion amount,
• wherein the pictures are captured by a fisheye optical system,
• the estimating includes:
• extracting first feature points from the first picture and second feature points from the second picture;
• matching the first feature points and the second feature points to estimate the motion amount indicating the displacement amount of the second picture with respect to the first picture, the first feature points and the second feature points being extracted in the extracting;
• and
• (i) transforming coordinates of the first feature points into three-dimensional coordinates of the first feature points which conform to a projection technique of the fisheye optical system and (ii) transforming coordinates of the second feature points into three-dimensional coordinates of the second feature points which conform to the projection technique of the fisheye optical system,
• the first feature points and the second feature points being extracted in the extracting, and
• the matching involves matching the three-dimensional coordinates of the first feature points and the three-dimensional coordinates of the second feature points to estimate the motion amount indicating the displacement amount of the second picture with respect to the first picture.

A non-transitory computer-readable recording medium for use in a computer, the recording medium having a computer program recorded thereon, the computer program causing the computer to correct displacement between pictures obtained by temporally-continued capturing, and the program causing the computer to execute:

• estimating a motion amount indicating a displacement amount of a second picture with respect to a first picture using feature points extracted from each of the first picture and the second picture, the second picture being captured temporally after the first picture;
• determining, using the feature points, whether or not correction is made using the motion amount estimated by the motion amount estimating unit;
• and
• correcting the displacement of the second picture with respect to the first picture using the motion amount so as to correct the displacement between the pictures, in the case where the determining involves determining that the correction is made using the motion amount,
• wherein the pictures are captured by a fisheye optical system,
• the estimating includes:
• extracting first feature points from the first picture and second feature points from the second picture;
• matching the first feature points and the second feature points to estimate the motion amount indicating the displacement amount of the second picture with respect to the first picture, the first feature points and the second feature points being extracted in the extracting;
• and
• (i) transforming coordinates of the first feature points into three-dimensional coordinates of the first feature points which conform to a projection technique of the fisheye optical system and (ii) transforming coordinates of the second feature points into three-dimensional coordinates of the second feature points which conform to the projection technique of the fisheye optical system,
• the first feature points and the second feature points being extracted in the extracting, and
• the matching involves matching the three-dimensional coordinates of the first feature points and the three-dimensional coordinates of the second feature points to estimate the motion amount indicating the displacement amount of the second picture with respect to the first picture.

An integrated circuit for correcting displacement between pictures obtained by temporally-continuous capturing, the integrated circuit comprising:

• a motion amount estimating unit configured to estimate, using feature points extracted from each of a first picture and a second picture, a motion amount indicating an amount of displacement of the second picture with respect to the first picture, the second picture being captured temporally after the first picture;
• a determining unit configured to determine, using the feature points, whether or not correction is made using the motion amount estimated by the motion amount estimating unit;
• and
• a picture correcting unit configured to correct the displacement of the second picture with respect to the first picture using the motion amount so as to correct the displacement between the pictures, in the case where the determining unit determines that the correction is made using the motion amount,
• wherein the pictures are captured by a fisheye optical system,
• the motion amount estimating unit includes:
• a feature point extracting unit configured to extract first feature points from the first picture and second feature points from the second picture;
• a feature point matching unit configured to match the first feature points and the second feature points to estimate the motion amount indicating the displacement amount of the second picture with respect to the first picture, the first feature points and the second feature points being extracted by the feature point extracting unit;
• and
• a feature point coordinate transforming unit configured to (i) transform coordinates of the first feature points into three-dimensional coordinates which conform to a projection technique of the fisheye optical system and (ii) transform coordinates of the second feature points into three-dimensional coordinates of the second feature points which conform to the projection technique of the fisheye optical system, the first feature points and the second feature points being extracted by the feature point extracting unit, and
• the feature point matching unit is configured to match the three-dimensional coordinates of the first feature points and the three-dimensional coordinates of the second feature points to estimate the motion amount indicating the displacement amount of the second picture with respect to the first picture.