Methods, systems, and media for generating an immersive light field video with a layered mesh representation
AssigneeGoogle Inc.
InventorsRyan Overbeck, Michael Joseph Broxton, John Flynn, Daniel William Erickson, Lars Peter Johannes Hedman, Matthew Nowicki DuVall, Jason Angelo Dourgarian, Jessica Lynn Busch, Matthew Stephen Whalen, Paul Debevec
Life of this patent
Estimating the term…
Abstract
Mechanisms for generating compressed images are provided. More particularly, methods, systems, and media for capturing, reconstructing, compressing, and rendering view-dependent immersive light field video with a layered mesh representation are provided.
Claims21
A method for generating compressed videos, the method comprising:
- obtaining a sequence of a plurality of images from a plurality of camera devices;
- generating, for each of the plurality of images in the sequence of the plurality of images, a multi-sphere image, wherein the multi-sphere image includes a series of concentric spherical shells each having an RGBA texture map that is generated based on the plurality of images obtained from the plurality of camera devices;
- generating a plurality of layered meshes corresponding to the sequence of the plurality of images by converting each multi-sphere image into a layered mesh, wherein the layered mesh is a polygonal mesh with a corresponding RGBA texture;
- consolidating the plurality of layered meshes and the RGBA textures associated with the plurality of layered meshes, wherein consolidating the plurality of layered meshes and the RGBA textures associated with the plurality of layered meshes comprises generating a sequence of texture-atlased images each corresponding to a Group of Pictures frame;
- generating a compressed video by compressing the stream of texture-atlased images;
- and
- storing the compressed video in association with an identifier of the sequence of the plurality of images.
A system for generating compressed videos, the system comprising:
- a hardware processor that is configured to:
- obtain a sequence of a plurality of images from a plurality of camera devices;
- generate, for each of the plurality of images in the sequence of the plurality of images, a multi-sphere image, wherein the multi-sphere image includes a series of concentric spherical shells each having an RGBA texture map that is generated based on the plurality of images obtained from the plurality of camera devices;
- generate a plurality of layered meshes corresponding to the sequence of the plurality of images by converting each multi-sphere image into a layered mesh, wherein the layered mesh is a polygonal mesh with a corresponding RGBA texture;
- consolidate the plurality of layered meshes and the RGBA textures associated with the plurality of layered meshes, wherein consolidating the plurality of layered meshes and the RGBA textures associated with the plurality of layered meshes comprises generating a sequence of texture-atlased images each corresponding to a Group of Pictures frame;
- generate a compressed video by compressing the stream of texture-atlased images;
- and
- store the compressed video in association with an identifier of the sequence of the plurality of images.
A non-transitory computer-readable medium containing computer executable instructions that, when executed by a processor, cause the processor to perform a method for generating compressed images, the method comprising:
- obtaining a sequence of a plurality of images from a plurality of camera devices;
- generating, for each of the plurality of images in the sequence of the plurality of images, a multi-sphere image, wherein the multi-sphere image includes a series of concentric spherical shells each having an RGBA texture map that is generated based on the plurality of images obtained from the plurality of camera devices;
- generating a plurality of layered meshes corresponding to the sequence of the plurality of images by converting each multi-sphere image into a layered mesh, wherein the layered mesh is a polygonal mesh with a corresponding RGBA texture;
- consolidating the plurality of layered meshes and the RGBA textures associated with the plurality of layered meshes, wherein consolidating the plurality of layered meshes and the RGBA textures associated with the plurality of layered meshes comprises generating a sequence of texture-atlased images each corresponding to a Group of Pictures frame;
- generating a compressed video by compressing the stream of texture-atlased images;
- and
- storing the compressed video in association with an identifier of the sequence of the plurality of images.
Dependencies are inferred from the claim text.