Back to results
US 12,112,427 B2Granted
High-definition real-time view synthesis
AssigneeSnap Inc.
InventorsNumair Khalil Ullah Khan, Gurunandan Krishnan Gorumkonda, Shree K. Nayar, Yicheng Wu
Life of this patent
Estimating the term…
Abstract
Images of a scene are received. The images represent viewpoints corresponding to the scene. A pixel map of the scene is computed based on the plurality of images. Multi-plane image (MPI) layers from the pixel map are extracted in real-time. The MPI layers are aggregated. The scene is rendered from a novel viewpoint based on the aggregated MPI layers.
Claims14
A method comprising:
- receiving a plurality of images captured from a reference viewpoint of a scene, each image of the plurality of images representing a view of the scene at a predefined depth from the reference viewpoint;
- generating a two-dimensional (2D) pixel map of the scene based on the plurality of images;
- determining one or more multi-plane image (MPI) layers from the 2D pixel map in real-time;
- dividing the one or more MPI layers into a plurality of MPI regions,
- wherein each MPI region of the plurality of MPI regions has a different set of depth planes than all other MPI regions of the plurality of MPI regions, and
- wherein a number of depth planes of the one or more MPI layers is larger than a number of depth planes of each MPI region of the plurality of MPI regions;
- constructing a textured three-dimensional mesh based on the plurality of MPI regions;
- rendering the scene from a novel viewpoint based on the textured three-dimensional mesh.
5 more elements
A computing apparatus, the computing apparatus comprising:
- a processor;
- and
- a memory storing instructions that, when executed by the processor, configure the apparatus to:
- receive a plurality of images captured from a reference viewpoint of a scene, each image of the plurality of images representing a view of the scene at a predefined depth from the reference viewpoint;
- generate a two-dimensional (2D) pixel map of the scene based on the plurality of images;
- determine one or more of multi-plane image (MPI) layers from the 2D pixel map in real-time;
- divide the one or more MPI layers into a plurality of MPI regions,
- wherein each MPI region of the plurality of MPI regions has a different set of depth planes than all other MPI regions of the plurality of MPI regions, and
- wherein a number of depth planes of the one or more MPI layers is larger than a number of depth planes of each MPI region of the plurality of MPI regions;
- construct a textured three-dimensional mesh based on the plurality of MPI regions;
- render the scene from a novel viewpoint based on the textured three-dimensional mesh.
8 more elements
A non-transitory computer-readable storage medium including instructions that, when processed by a computer, configure the computer to perform:
- receiving a plurality of images captured from a reference viewpoint of a scene, each image of the plurality of images representing a view of the scene at a predefined depth from the reference viewpoint;
- generating a two-dimensional (2D) pixel map of the scene based on the plurality of images;
- determining one or more of multi-plane image (MPI) layers from the 2D pixel map in real-time;
- dividing the one or more MPI layers into a plurality of MPI regions,
- wherein each MPI region of the plurality of MPI regions has a different set of depth planes than all other MPI regions of the plurality of MPI regions, and
- wherein a number of depth planes of the one or more MPI layers is larger than a number of depth planes of each MPI region of the plurality of MPI regions;
- constructing a textured three-dimensional mesh based on the plurality of MPI regions;
- rendering the scene from a novel viewpoint based on the textured three-dimensional mesh.
5 more elements
Dependencies are inferred from the claim text.