Method and apparatus for three-dimensional scene processing using parallax geometry of pairs of points

Method for image processing to recover scene structure comprising the steps of:

• a) receiving a plurality of two dimensional images representative of a scene, wherein each image is taken from an arbitrary camera position;
• b) computing a planar homography aligning a surface in the scene among the plurality of images;
• c) computing parallax vectors with respect to respective corresponding points in each of two images of the plurality of images;
• d) using the parallax vectors and additional information in the plurality of images, computing a parallax rigidity constraint for the plurality of images, the parallax rigidity constraint being independent of any epipole;
• e) applying the parallax rigidity constraint to a further plurality of points within the plurality of images in order to generate information representative of whether a given point within the plurality of images is consistent with the parallax structure constraint;
• and
• f) using the generated information to perform recovery of scene structure related to the received plurality of images.

Method for image processing to generate new views comprising the steps of:

• a) receiving a plurality of two dimensional images representative of a scene, each image taken from an arbitrary camera position;
• b) computing a planar homography aligning a surface in the scene among the plurality of images;
• c) computing parallax vectors with respect to respective corresponding points in each of two images of the plurality of images;
• d) using the parallax vectors and additional information in the plurality of images, computing a parallax rigidity constraint for the plurality of images, the parallax rigidity constraint being independent of any epipole;
• e) applying the parallax rigidity constraint to a further plurality of points within the plurality of images in order to generate information representative of whether a given point within the plurality of images is consistent with the parallax rigidity constraint;
• and
• f) using the generated information to perform new view generation related to and based on the received plurality of images.

Apparatus for image processing comprising:

• a source of two dimensional images representative of a scene;
• a computer processor for processing the two dimensional images including:
• means for receiving a plurality of two dimensional images representative of a scene;
• means for computing a planar homography aligning a surface in the scene among the plurality of images;
• means for computing parallax vectors with respect to the planar homography of a plurality of points among the plurality of images;
• means for computing a parallax-related constraint directly from the positions of the plurality of points and the parallax vectors, the parallax-related constraint being independent of any epipole;
• means for applying the parallax-related constraint to a further plurality of points within the plurality of images in order to generate information representative of whether a given point within the plurality of images is consistent with the parallax-related constraint;
• means for using the generated information for an image processing task related to the received plurality of images and generating an output related thereto;
• and
• an output device for presenting the output of the image processing task.

A computer-readable medium having stored thereon a plurality of instructions, the plurality of instructions including instructions which, when executed by a processor, cause the processor to perform the steps of:

• a) receiving, in digital form, a plurality of two dimensional images representative of a scene;
• b) computing a planar homography aligning a surface in the scene among the plurality of images;
• c) computing parallax vectors with respect to the planar homography for a plurality of points among the plurality of images;
• d) computing a parallax-related constraint directly from the positions of the plurality of points and the parallax vectors, the parallax-related constraint being independent of any epipole;
• e) applying the parallax-related constraint to a further plurality of points within the plurality of images in order to generate information representative of whether a given point within the plurality of images is consistent with the parallax-related constraint;
• and
• f) using the generated information for an image processing task related to the received plurality of images.

Method for image processing comprising the steps of:

• a) receiving a plurality of two dimensional images representative of a scene, the images taken from arbitrary camera positions;
• b) computing a parallax-related constraint for a plurality of points within the plurality of images including the steps of:
• i) warping the plurality of images to compensate for effects of scaling and rotation around the camera positions;
• ii) computing parallax vectors for the plurality of points;
• and
• iii) computing the parallax-related constraint directly from the positions of the plurality of points and the parallax vectors, the parallax-related constraint being independent of any epipole;
• c) applying the parallax-related constraint to a further plurality of points within the plurality of images in order to generate information representative of whether a given point within the plurality of images is consistent with the parallax-related constraint;
• and
• d) using the generated information for an image processing task related to the received plurality of images.