Image view synthesis using a three-dimensional reference model

A method for synthesizing a novel image view, the method comprising using a computing device to perform steps for:

• registering an image to at least one reference model;
• selecting a source block of visual data from the image with regard to a destination block of the reference model based on a source depth associated with the source block and a destination depth associated with the destination block;
• wherein each pixel of each source block includes two or more depth levels, with each depth level having a separate color assignment;
• wherein the color assignment of a frontmost layer of each pixel is the same color as a corresponding pixel of the image, and wherein a color assignment of each subsequent depth layer for each pixel is synthesized via multi-resolution texture synthesis;
• applying each pixel of the source block of visual data from the image to the destination block of the reference model using a computed distance of the reference model to a position of a camera used to capture the image, and using the pixel color assignment from the depth layer closest to the computed distance to produce an image-augmented model;
• wherein the step for applying comprises a step for decreasing a proportional amount of the source block of visual data from the image that is applied to the destination block of the reference model as a novel perspective viewpoint corresponding to a novel image view deviates further from a capture perspective viewpoint corresponding to the image;
• and
• synthesizing the novel image view using the image-augmented model.

A system for synthesizing a novel image view, comprising:

• an image registration device that causes a computer to register an image to at least one reference model;
• a source block selector device that causes a computer to select a source block of one or more pixels of visual data from the image with regard to a destination block of the reference model based on a source depth associated with the source block and a destination depth associated with the destination block;
• wherein each pixel of each source block includes two or more depth levels, with each depth level having a separate color assignment;
• wherein the color assignment of a frontmost layer of each pixel is the same color as a corresponding pixel of the image, and wherein a color assignment of each subsequent depth layer for each pixel is synthesized via multi-resolution texture synthesis;
• a visual image data application device that causes a computer to apply the source block of visual data from the image to the destination block of the reference model using a computed distance of the reference model to a camera position at which the image was originally captured, and using the pixel color assignment from the depth layer closest to the computed distance to produce an image-augmented model;
• wherein applying the source block of visual data further comprises decreasing a proportional amount of the visual data that is applied to the destination block as a novel perspective viewpoint corresponding to a novel image view deviates further from a capture perspective viewpoint corresponding to the image;
• and
• an image view synthesizer device that causes a computer to synthesize the novel image view using the image-augmented model.