Back to results
US 7,301,548 B2Granted
System and method for whiteboard scanning to obtain a high resolution image
AssigneeMicrosoft Corporation
InventorsZhengyou Zhang, Li-wei He
Life of this patent
Estimating the term…
Abstract
This invention is directed toward a system and method for scanning a scene or object such as a whiteboard, paper document or similar item. More specifically, the invention is directed toward a system and method for obtaining a high-resolution image of a whiteboard or other object with a low-resolution camera. The system and method of the invention captures either a set of snapshots with overlap or a continuous video sequence, and then stitches them automatically into a single high-resolution image. The stitched image can finally be exported to other image processing systems and methods for further enhancement.
Claims20
A computer-implemented process for converting the contents of a planar object into a high-resolution image, comprising the process actions of:
- acquiring a sequence of images of portions of a planar object which have been captured in a prescribed pattern and wherein each subsequent image overlaps a previous image in said pattern;
- extracting points of interest in each image;
- matching said points of said interest between each pair of successive images thereby creating a set of point matches;
- computing a projective mapping between each pair of successive images using a least median squares technique which detects both false point matches and simultaneously estimates a homography matrix in order to determine corresponding pixel locations in the images, wherein said computing a projective mapping comprises,
- (a) inputting a first image and a second image;
- (b) drawing m random subsamples of a specified number of at least four different point matches of said set of point matches;
- (c) for each subsample J, computing a homograph matrix Hj;
- (d) for each Hj, determining the median of the squared residuals, denoted by Mj, with respect to the whole set of point matches, where the squared residual for match i is given by ∥m21−{circumflex over (m)}1i∥2 where {circumflex over (m)}1i is point m1i transferred to the second image by Hj;
- (e) retaining the estimate Hj for which Mj is minimal among all m Mj's;
- (f) computing a robust standard deviation estimate {circumflex over (σ)};
- (g) declaring a point match as a false match if its residual is larger than k {circumflex over (σ)}, where k is set to a prescribed value;
- (h) discarding the false matches and re-estimating H by minimizing the sum of squared errors
- ∑
- i
-
-
- m
- 2
-
- i
- -
- m
- ^
- 1
-
- i
-
- 2
- where the summation is over all good matches;
- and
- (i) repeating process actions (a) through (h) until all images of the sequence of images have been processed;
- and
- generating a composite image from said sequence of images using said projective mapping.
30 more elements
A system for converting markings on a planar object into a high resolution image, the system comprising:
- a general purpose computing device;
- and
- a computer program comprising program modules executable by the computing device, wherein the corrupting device is directed by the program modules of the computer program to,
- acquire a sequence of images of portions of a planar object having been captured in a prescribed pattern, each subsequent image overlapping a previous image in said pattern;
- extract points of interest in each image in said sequence;
- match said points of said interest between two successive images in said sequence thereby creating a set of point matches;
- compute a projective mapping between each set of two successive images in said sequence of images using a east median squares technique which detects both false point matches and simultaneously estimates a homography matrix in order to determine corresponding pixel locations in the images of each set, therein said computing a projective mapping comprises,
- (a) inputting a first image and a second image;
- (b) drawing m random subsamples of a specified number of at least four different point matches of said set of point matches;
- (c) for each subsample J, computing a homography matrix HJ;
- (d) for each HJ, determining the median of th squared residuals, denoted by MJ, with respect to the whole set of point matches, where the squared residual for match j is given by ∥m2i−{circumflex over (m)}1i∥2 where {circumflex over (m)}1i is point m1i transferred to the second image by HJ;
- (e) retaining the estimate HJ for which MJ is minimal among all m MJ's;
- (f) computing a robust standard deviation estimated {circumflex over (σ)};
- (g) declaring a point match as a false match if its residual is larger than k {circumflex over (σ)}, where k is set to a prescribed value;
- (h) discarding the false matches and re-estimating H by minimizing the sum of squared errors
- ∑
- i
-
-
- m
- 2
-
- i
- -
- m
- ^
- 1
-
- i
-
- 2
- where the summation is over all good matches;
- and
- (i) repeating process actions (a) through (h) until all images of the sequence of images have been processed;
- and
- generate a composite image from said images using said projective mapping.
33 more elements
A computer-readable medium having computer-executable instructions for converting a series of low resolution images of portions of a planar object into a high resolution image of said object, said computer executable instructions causing a computer to execute the method comprising:
- acquiring a series of images of the depicting portions of the same scene:
- extracting points of interest in each image of said series of images;
- matching said points of interest in each image of said series of images with the image preceding said image in said series of images thereby creating a set of point matches;
- using a least median squares technique which detects both false point matches and simultaneously estimates a homography matrix to calculate a homography between each image of said series of images with the image preceding said image in said series of images, wherein said calculating a homography comprises,
- (a) inputting a first image and a second image;
- (b) drawing m random subsamples of a specified number of at least for different point matches of said set of point matches;
- (c) for each subsample J, computing a homography matrix Hj;
- (d) for each HJ, determining the median of the squared residuals, denoted by MJ, with respect to the whole set of point matches, where the squared residual for match i is given by ∥m2i−{circumflex over (m)}1i is point m1i transferred to the second image by HJ;
- (e) retaining the estimate HJ for which MJ is minimal among all m MJ's;
- (f) computing a robust standard deviation estimate {circumflex over (σ)};
- (g) declaring a point match as a false match if its residual is larger than k{circumflex over (σ)}, where k is set to a prescribed value;
- (h) discarding the false matches and re-estimating H by minimizing the sum of squared errors
- ∑
- i
-
-
- m
- 2
-
- i
- -
- m
- ^
- 1
-
- i
-
- 2
- matches;
- and
- (i) repeating process actions (a) through (h) until all images of the sequence of images have been processed;
- and
- stitching each image in said series of images together using said homographies to create a composite image.
30 more elements
Dependencies are inferred from the claim text.