View synthesis from plural images using a trifocal tensor data structure in a multi-view parallax geometry

A process for synthesizing an image of a scene in accordance with a new viewpoint from a pair of existing images of the scene taken from respective initial viewpoints, comprising:

• choosing a planar surface visible in said two existing images and transforming said existing images relative to one another so as to bring said planar surface into perspective alignment in said at least two existing images;
• selecting a reference frame and at least two scene points in said existing image, and determining the projection on the reference frame of parallax vectors between the two images of said two scene points;
• specifying the new viewpoint by specifying the projection on the reference frame of said two scene points as viewed from the new viewpoint;
• computing parallax vectors between said initial viewpoints and said new viewpoint of said projections of said two scene points;
• computing from said parallax vectors of said three scene points a trifocal tensor data structure;
• using said trifocal tensor data structure to compute from pixels in said existing image a new view image;
• and
• wherein the step of computing said trifocal tensor data structure comprises computing three epipoles in said reference frame corresponding to said three viewpoints and computing the elements of said trifocal data structure from said three epipoles.

Apparatus for use in an image processing system for synthesizing a new image representing a new viewpoint of a scene from at least two existing images of the scene taken from different respective viewpoints, said apparatus comprising:

• a processor;
• a memory having executable instructions stored therein;
• wherein the processor, in response to the instructions stored in the memory:
• refers to a chosen planar surface visible in said at least two existing images;
• transforms said at least two existing images relative to one another so as to bring said chosen planar surface into perspective alignment in said at least two existing images;
• refers to a chosen reference frame and computes parallax vectors between said two images of the projection of common scene points on said reference frame;
• refers to a specified image parameter on said reference frame of said new viewpoint, and constructs from said image parameter and from said parallax vectors a trifocal tensor data structure;
• computes scene points in said new view from the scene points in said existing images using said trifocal tensor data structure, to produce said synthesized new view image;
• and
• wherein said image parameter of said new view comprises information sufficient, together with said parallax vectors, to deduce:
• (a) a trifocal ratio in the reference frame and (b) one epipole between the new viewpoint and one of said first and second viewpoints.

Apparatus for use in an image processing system for synthesizing an image of a scene in accordance with a new viewpoint from a pair of existing images of the scene taken from respective initial viewpoints, said apparatus comprising:

• a processor;
• a memory having executable instructions store therein;
• wherein the processor, in response to the instructions stored in the memory:
• refers to a chosen planar surface visible in said two existing images and transforms said existing images relative to one another so as to bring said planar surface into perspective alignment in said at least two existing images;
• refers to a selected reference frame and at least two scene points in said existing image, and determines the projection on the reference frame of parallax vectors between the two images of said two scene points;
• refers to a new viewpoint specified by a projection on the reference frame of said two scene points as viewed from the new viewpoint;
• computes parallax vectors between said initial viewpoints and said new viewpoint of said projections of said two scene points;
• computes from said parallax vectors of said three scene points a trifocal tensor data structure;
• compute from pixels in said existing image a new view image using said trifocal tensor data structure;
• and
• wherein said processor computes said trifocal tensor data structure in that said processor computes three epipoles in said reference frame corresponding to said three viewpoints and computes the elements of said trifocal tensor data structure from said three epipoles.