Optical flow estimation method and image synthesis method
AssigneesSanyo Electric Co., Ltd., Quantapoint, Inc.
InventorsNaoki Chiba, Takeo Kanade, Hiroshi Kano
Life of this patent
Estimating the term…
Abstract
Optical flow is calculated for each sub-block on the basis of two images, the sub-blocks are then partitioned into a region having low confidence and a region having high confidence depending on the reliability of the optical flow, and the optical flow in the region having low confidence is filled by dilation processing using the optical flow in its surrounding regions.
Claims12
An optical flow estimation method so adapted as to calculate optical flow for each sub-block on the basis of two images, partition the sub-blocks into a region having low confidence and a region having high confidence depending on the reliability of the optical flow, and fill the optical flow in the region having low confidence by dilation processing using the optical flow in its surrounding regions
- wherein confidence γ is found, upon calculating from a 2×2 matrix of coefficients G in the following equation (a) having as elements the squares of differentials in the vertical and horizontal directions of an image region ω and its eigenvalues, by the following equation (b) from the smaller eigenvalue λmin and a gray scale residual E between the regions after tracking:
- G = ∑ p ∈ ω g ( p ) g ( p ) T ( a ) γ = λ min E ( b )
An image synthesis method for synthesizing a first image and a second image without a seam, comprising:
- the first step of extracting an overlapped portion of the first image and the second image;
- the second step of extracting from the overlapped portion of the first image a plurality of partial images which are effective for tracking by optical flow as feature points;
- the third step of tracking a position, on the second image, of each of the feature points on the first image on the basis of the optical flow;
- the fourth step of finding a planar projective transform matrix for finding, on the basis of each of the feature points on the first image and the corresponding position on the second image which is found in the third step, a point, corresponding to the point on the first image, on the second image;
- and
- the fifth step of synthesizing the first image and the second image such that the corresponding points on the first image and the second image coincide with each other on the basis of the obtained planar projective transform matrix.
An image synthesizer for synthesizing a first image and a second image without a seam, comprising:
- first means for extracting an overlapped portion of the first image and the second image;
- second means for extracting from the overlapped portion of the first image a plurality of partial images which are effective for tracking by optical flow as feature points;
- third means for tracking a position, on the second image, of each of the feature points on the first image on the basis of the optical flow;
- fourth means for finding a planar projective transform matrix for finding, on the basis of each of the feature points on the first image and the corresponding position on the second image which is found in the third step, a point, corresponding to the point on the first image, on the second image;
- and
- fifth means of synthesizing the first image and the second image such that the corresponding points on the first image and the second image coincide with each other on the basis of the obtained planar projective transform matrix.
A computer readable recording medium on which an image synthesis program for synthesizing a first image and a second image without a seam is recorded, wherein
- the image synthesis program comprises
- the first step of extracting an overlapped portion of the first image and the second image,
- the second step of extracting from the overlapped portion of the first image a plurality of partial images which are effective for tracking by optical flow as feature points,
- the third step of tracking a position, on the second image, of each of the feature points on the first image on the basis of the optical flow,
- the fourth step of finding a planar projective transform matrix for finding, on the basis of each of the feature points on the first image and the corresponding position on the second image which is found in the third step, a point, corresponding to the point on the first image, on the second image, and
- the fifth step of synthesizing the first image and the second image such that the corresponding points on the first image and the second image coincide with each other on the basis of the obtained planar projective transform matrix.
A digital camera in which an image synthesis program for synthesizing a first image and a second image without a seam is incorporated, wherein
- the image synthesis program comprises
- the first step of extracting an overlapped portion of the first image and the second image;
- the second step of extracting from the overlapped portion of the first image a plurality of partial images which are effective for tracking by optical flow as feature points;
- the third step of tracking the position, on the second image, of each of the feature points on the first image on the basis of the optical flow;
- the fourth step of finding a planar projective transform matrix for finding, on the basis of each of the feature points on the first image and the corresponding position on the second image which is found in the third step, a point, corresponding to the point on the first image, on the second image;
- and
- the fifth step of synthesizing the first image and the second image such that the corresponding points on the first image and the second image coincide with each other on the basis of the obtained planar projective transform matrix.
A printer in which an image synthesis program for synthesizing a first image and a second image without a seam is incorporated, wherein
- the image synthesis program comprises
- the first step of extracting an overlapped portion of the first image and the second image;
- the second step of extracting from the overlapped portion of the first image a plurality of partial images which are effective for tracking by optical flow as feature points;
- the third step of tracking a position, on the second image, of each of the feature points on the first image on the basis of the optical flow;
- the fourth step of finding a planar projective transform matrix for finding, on the basis of each of the feature points on the first image and a corresponding position on the second image which is found in the third step, a point, corresponding to the point on the first image, on the second image;
- and
- the fifth step of synthesizing the first image and the second image such that the corresponding points on the first image and the second image coincide with each other on the basis of the obtained planar projective transform matrix.
Dependencies are inferred from the claim text.