Three-Dimensional Mesh Compression Using a Video Encoder
AssigneeApple Inc.
InventorsKhaled Mammou, Alexandros Tourapis, Jungsun Kim
Life of this patent
Estimating the term…
Abstract
A system comprises an encoder configured to compress and encode data for a three-dimensional mesh using a video encoding technique. To compress the three-dimensional mesh, the encoder determines sub-meshes and for each sub-mesh: texture patches and geometry patches. Also the encoder determines patch connectivity information and patch texture coordinates for the texture patches and geometry patches. The texture patches and geometry patches are packed into video image frames and encoded using a video codec. Additionally, the encoder determines boundary stitching information for the sub-meshes. A decoder receives a bit stream as generated by the encoder and reconstructs the three-dimensional mesh.
Claims20
A non-transitory computer-readable medium storing program instructions that, when executed by one or more processors, cause the one or more processors to:
- determine, for a three-dimensional mesh, a plurality of patches, wherein the patches comprise:
- one or more attribute or texture patches;
- and
- for each of a plurality of sub-meshes of the three-dimensional mesh, a geometry patch that indicates depth values for vertices of the respective sub-mesh;
- pack two-dimensional patch images for the determined patches into a plurality of two-dimensional image frames;
- determine patch connectivity information and patch texture or attribute coordinates for the patches;
- determine boundary stitching information to be used to join the sub-meshes back into the three-dimensional mesh;
- and
- encode the determined patches, the determined patch connectivity and patch texture or attribute coordinates, and the determined boundary stitching information,
- wherein the determined patches are encoded via a video-based encoder as packed two-dimensional patch images packed into the plurality of two-dimensional image frames.
A device, comprising:
- a memory storing program instructions;
- and
- one or more processors, wherein the program instructions, when executed by the one or more processors, cause the one or more processors to:
- determine, for a three-dimensional mesh, a plurality of patches, wherein the patches comprise:
- one or more attribute or texture patches;
- and
- for each of a plurality of sub-meshes of the three-dimensional mesh, a geometry patch that indicates depth values for vertices of the respective sub-mesh;
- pack two-dimensional patch images for the determined patches into a plurality of two-dimensional image frames;
- determine patch connectivity information and patch texture or attribute coordinates for the patches;
- determine boundary stitching information to be used to join the sub-meshes back into the three-dimensional mesh;
- and
- encode the determined patches, the determined patch connectivity and patch texture or attribute coordinates, and the determined boundary stitching information.
A non-transitory computer-readable medium storing program instructions that, when executed by one or more processors, cause the one or more processors to:
- receive a compressed bit-stream for a three-dimensional mesh, wherein the bit stream comprises:
- one or more video encoded image frames comprising patch images packed into the video encoded image frames, wherein the patch images comprise:
- geometry patches indicating depth values for vertices of a plurality of sub-meshes of the three-dimensional mesh;
- and
- texture or attribute patches for the plurality of sub-meshes of the three-dimensional mesh;
- patch connectivity information and patch texture or attribute coordinates;
- and
- boundary stitching information;
- video decode the video encoded image frames;
- determine, based on the patch texture or attribute coordinates and corresponding geometry patches for respective ones of the sub-meshes, locations of vertices of the respective ones of the sub-meshes in three-dimensional space;
- and
- reconstruct the three-dimensional mesh by applying the patch connectivity information to the vertices, wherein the boundary stitching information is further used to merge vertices of adjacent sub-meshes that correspond to a same vertex in the reconstructed three-dimensional mesh.
-22. (canceled)
Dependencies are inferred from the claim text.