Video-based point cloud compression with variable patch scaling
AssigneeApple Inc.
InventorsJungsun Kim, Khaled Mammou, Alexandros Tourapis
Life of this patent
Estimating the term…
Abstract
A system comprises an encoder configured to compress attribute information and/or spatial information for volumetric visual content and/or a decoder configured to decompress compressed attribute and/or spatial information for the volumetric visual content. The encoder is configured to convert a 3D representation of the visual volumetric content into a 2D image based representation. The encoder is further configured to scale the patch in 2D space independent of any scaling in 3D space. Auxiliary information is signaled for use in identifying 2D scaled or unscaled patches in an image frame, mapping the patches into 3D space, and adjusting for any scaling factors applied at the encoder.
Claims20
A non-transitory, computer-readable, medium storing program instructions that, when executed by one or more processors, cause the one or more processors to:
- generate 2D patch images for three-dimensional (3D) visual volumetric content;
- scale one or more of the 2D patch images in 2D space, without requiring a corresponding 3D bounding box for the 2D patch image to be scaled by a same scaling factor in a same coordinate direction;
- pack the scaled one or more 2D patch images into one or more two-dimensional (2D) image frames;
- generate auxiliary information indicating:
- respective 2D bounding boxes in the 2D image frames for the 2D patch images that have been scaled;
- respective locations or characteristics of 3D bounding boxes in a 3D reconstructed version of the 3D visual volumetric content at which patches corresponding to the 2D patch images are to be located;
- and
- an indication of the one or more scaling factors that have been applied in the 2D space;
- and
- encode the one or more 2D image frames and the auxiliary information to generate a compressed version of the 3D visual volumetric content.
A device, comprising
- a memory storing program instructions;
- and
- one or more processors, wherein the program instructions, when executed by the one or more processors, cause the one or more processors to:
- generate 2D patch images for three-dimensional (3D) visual volumetric content;
- scale one or more of the 2D patch images in 2D space, without requiring a corresponding 3D bounding box for the 2D patch image to be scaled by a same scaling factor in a same coordinate direction;
- pack the scaled one or more 2D patch images into one or more two-dimensional (2D) image frames;
- generate auxiliary information indicating:
- respective 2D bounding boxes in the 2D image frames for the 2D patch images that have been scaled;
- respective locations or characteristics of 3D bounding boxes in a 3D reconstructed version of the 3D visual volumetric content at which patches corresponding to the 2D patch images are to be located;
- and
- an indication of the one or more scaling factors that have been applied in the 2D space;
- and
- encode the one or more 2D image frames and the auxiliary information to generate a compressed version of the 3D visual volumetric content.
A method, comprising:
- receiving 2D video images that comprise patch image data for compressed visual volumetric content, wherein at least some of the patch images have been scaled by one or more scaling factors that are not uniform for all of the patch images in a given 2D video image frame;
- receiving auxiliary information for patches of the compressed visual volumetric content packed into the 2D video images, wherein the auxiliary information indicates the one or more scaling factors;
- determining based on the auxiliary information, portions of the 2D video images that correspond to respective patches of the compressed visual volumetric content;
- and
- reconstructing a 3D representation of the visual volumetric content based on patches extracted from the 2D video images.
Dependencies are inferred from the claim text.