Image processing method and apparatus for stereoscopic images of nearby object in binocular camera system of parallel axis type

An image processing method of processing a stereo image to impart a three-dimensional effect on a nearby object in a stereoscopic image system using a stereo camera of a parallel axis type including a left camera and a right camera, the stereo image including a left image obtained by the left camera and a right image obtained by the right camera, the method comprising:

• calculating a first horizontal rotation matrix for adjusting a first horizontal parallax of the left image and a second horizontal rotation matrix for adjusting a second horizontal parallax of the right image;
• calculating a geometric transformation matrix for the stereo image based on each camera characteristic matrix representing lens characteristics of the stereo camera, the first horizontal rotation matrix, and the second horizontal rotation matrix;
• and
• generating a new stereo image in which the first horizontal parallax and the second horizontal parallax are adjusted by performing image warping for the stereo image captured by the stereo camera based on the calculated geometric transformation matrix,
• wherein the geometric transformation matrix is calculated by using the following equations,
• ML=KLRyT(θ)KL−1 and MR=KRRyT(−θ)RLRKR−1, and
• wherein ML is a geometric transformation matrix for the left image among the stereo image captured by the stereo camera, MR is a geometric transformation matrix for the right image among the stereo image captured by the stereo camera, KL is a camera characteristic matrix representing lens characteristics of the left camera, KR is a camera characteristic matrix representing lens characteristics of the right camera, Ry(θ) is the first horizontal rotation matrix for adjusting the first horizontal parallax for the left image, Ry(−θ) is the second horizontal rotation matrix for adjusting the second horizontal parallax for the right image, RLR is a rotation matrix from a left camera coordinate system to a right camera coordinate system, a superscript −1 means an inverse matrix, and a superscript T means transpose of a matrix.

An image processing apparatus for processing a stereo image to impart a three-dimensional effect on a nearby object in a stereoscopic image system using a stereo camera of a parallel axis type including a left camera and a right camera, the method comprising:

• an input unit configured to input the stereo image captured by the stereo camera;
• an image processing unit configured to adjust vertical parallax and horizontal parallax of the stereo image, the horizontal parallax including a first horizontal parallax of a left image obtained by the left camera and a second horizontal parallax of a right image obtained by the right camera;
• and
• an output unit configured to output the stereo image in which the vertical parallax and the horizontal parallax are adjusted by the image processing unit,
• wherein the image processing unit is configured to:
• calculate a vertical rotation matrix for eliminating vertical parallax of the stereo image by using the following equation,
• R
• rect
• =
• [
• e
• x
• T
• e
• y
• T
• e
• z
• T
• ]
• ,
• wherein
• e
• x
• =
• t
• LR
• ❘
• "\[LeftBracketingBar]"
• t
• LR
• ❘
• "\[RightBracketingBar]"
• ,
• wherein
• e
• y
• =
• 1
• t
• x
• 2
• +
• t
• y
• 2
• [
• -
• t
• y
• t
• x
• 0
• ]
• T
• ,
• wherein ez=ex×ey, and
• wherein
• tLR=[txtytz]T indicates a translation vector from a left camera coordinate system to a right camera coordinate system,
• ex indicates a unit vector in the x-axis direction of a new left camera coordinate system,
• ey indicates a unit vector in the y-axis direction of the new left camera coordinate system, and
• ez indicates a unit vector in the z-axis direction of the new left camera coordinate system, and
• wherein the stereo image includes a left image obtained by the left camera and a right image obtained by the right camera;
• calculate a first horizontal rotation matrix for adjusting the first horizontal parallax of the left image and a second horizontal rotation matrix for adjusting the second horizontal parallax of the right image;
• calculate a geometric transformation matrix for the stereo image based on each camera characteristic matrix representing lens characteristics of the stereo camera, the vertical rotation matrix, the first horizontal rotation matrix, and the second horizontal rotation matrix;
• and
• generate a new stereo image in which the vertical parallax is eliminated and the first horizontal parallax and the second horizontal parallax are is adjusted by performing image warping for the stereo image captured by the stereo camera based on the calculated geometric transformation matrix to provide to the output unit.