# US 11,908,169 B2 · Dense mesh compression

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,908,169 B2
- Document ID: US11908169B2
- Status: Granted
- Kind: B2
- Application number: 17/317,650
- Filed: 2021-05-11
- Published: 2022-04-07
- Granted: 2024-02-20
- Earliest claimed: 2020-10-01
- Assignees: Sony Group Corporation
- Applicants: Sony Group Corporation
- Inventors: Danillo Graziosi
- Web page: https://patentgrep.com/patent/US11908169B2

## CPC

- G06T 9/001: {Model-based coding, e.g. wire frame}
- G06T 17/205: {Re-meshing}
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- H04N 19/597: specially adapted for multi-view video sequence encoding
- G06T 2219/2012: Colour editing, changing, or manipulating; Use of colour codes
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method of compressing meshes using a projection-based approach, leveraging and expanding the tools and syntax generated for projection-based volumetric content compression is described. The mesh is segmented into surface patches, with the difference that the segments follow the connectivity of the mesh. The dense mesh compression utilizes 3D surface patches to represent connected triangles on a mesh surface and groups of vertices to represent triangles not captured by surface projection. Each surface patch (or 3D patch) is projected to a 2D patch, whereby for the mesh, the triangle surface sampling is similar to a common rasterization approach. For each patch, position and connectivity of the projected vertices are kept. The sampled surface resembles a point cloud and is coded with the same approach used for point cloud compression. The list of vertices and connectivity per patch is encoded, and the data is sent with the coded point cloud data.

## Claims

### Claim 1

A method comprising:
performing mesh voxelization on an input mesh;
implementing patch generation which segments the mesh into patches including a surface mesh and unprojected triangles;
performing mesh coding on the surface mesh;
performing triangle coding on the unprojected triangles; and
generating a Visual Volumetric Video-based Compression (V3C) bitstream based on the mesh coding and the triangle coding, wherein triangle coding includes determining if a bounding box of connected components is smaller than a pre-defined area, the unprojected triangles are moved to a separate list for independent triangle coding, and the unprojected triangles are not rasterized, but coded as vertices with associated color per vertex; otherwise, each triangle is projected to a patch, and if a projected position of a vertex is already occupied, the triangle is encoded in another patch, so the triangle is put in a missing triangles list to be processed again later.

### Claim 2 (depends on 1)

The method of claim 1 wherein mesh coding includes determining and storing a positions of projected vertices along with connectivity of the projected vertices.

### Claim 3 (depends on 1)

The method of claim 1 wherein triangle coding includes coding triangles not captured during surface projection.

### Claim 4 (depends on 3)

The method of claim 3 wherein triangle coding includes coding the triangles using only vertex position and color vertex information.

### Claim 5 (depends on 1)

The method of claim 1 wherein triangle coding utilizes a triangle patch data unit.

### Claim 6 (depends on 5)

The method of claim 5 wherein the triangle patch data unit stores a triangle as a separate triangle, a triangle strip or a triangle fan.

### Claim 7 (depends on 1)

The method of claim 1 wherein triangle coding further includes a color expansion implementation which writes a color of a triangle as c0 c1 c2; cm cm cm; c0 c1 c2, where cm is a color value of a centroid position of the triangle.

### Claim 8 (depends on 7)

The method of claim 7 wherein triangle coding further includes color compression.

### Claim 9

An apparatus comprising:
a non-transitory memory for storing an application, the application for:
performing mesh voxelization on an input mesh;
implementing patch generation which segments the mesh into patches including a surface mesh and unprojected triangles;
performing mesh coding on the surface mesh;
performing triangle coding on the unprojected triangles; and
generating a Visual Volumetric Video-based Compression (V3C) bitstream based on the mesh coding and the triangle coding, wherein triangle coding includes determining if a bounding box of connected components is smaller than a pre-defined area, the unprojected triangles are moved to a separate list for independent triangle coding, and the unprojected triangles are not rasterized, but coded as vertices with associated color per vertex; otherwise, each triangle is projected to a patch, and if a projected position of a vertex is already occupied, the triangle is encoded in another patch, so the triangle is put in a missing triangles list to be processed again later; and
a processor coupled to the memory, the processor configured for processing the application.

### Claim 10 (depends on 9)

The apparatus of claim 9 wherein mesh coding includes determining and storing a positions of projected vertices along with connectivity of the projected vertices.

### Claim 11 (depends on 9)

The apparatus of claim 9 wherein triangle coding includes coding triangles not captured during surface projection.

### Claim 12 (depends on 11)

The apparatus of claim 11 wherein triangle coding includes coding the triangles using only vertex position and color vertex information.

### Claim 13 (depends on 9)

The apparatus of claim 9 wherein triangle coding utilizes a triangle patch data unit.

### Claim 14 (depends on 13)

The apparatus of claim 13 wherein the triangle patch data unit stores a triangle as a separate triangle, a triangle strip or a triangle fan.

### Claim 15 (depends on 9)

The apparatus of claim 9 wherein triangle coding further includes a color expansion implementation which writes a color of a triangle as c0 c1 c2; cm cm cm; c0 c1 c2, where cm is a color value of a centroid position of the triangle.

### Claim 16 (depends on 15)

The apparatus of claim 15 wherein triangle coding further includes color compression.

### Claim 17

A system comprising:
one or more cameras for acquiring three dimensional content;
an encoder for encoding the three dimensional content:
performing mesh voxelization on an input mesh of the three dimensional content;
implementing patch generation which segments the mesh into patches including a surface mesh and unprojected triangles;
performing mesh coding on the surface mesh;
performing triangle coding on the unprojected triangles; and
generating a Visual Volumetric Video-based Compression (V3C) bitstream based on the mesh coding and the triangle coding, wherein triangle coding includes determining if a bounding box of connected components is smaller than a pre-defined area, the unprojected triangles are moved to a separate list for independent triangle coding, and the unprojected triangles are not rasterized, but coded as vertices with associated color per vertex; otherwise, each triangle is projected to a patch, and if a projected position of a vertex is already occupied, the triangle is encoded in another patch, so the triangle is put in a missing triangles list to be processed again later.

### Claim 18 (depends on 17)

The system of claim 17 wherein mesh coding includes determining and storing a positions of projected vertices along with connectivity of the projected vertices.

### Claim 19 (depends on 17)

The system of claim 17 wherein triangle coding includes coding triangles not captured during surface projection.

### Claim 20 (depends on 19)

The system of claim 19 wherein triangle coding includes coding the triangles using only vertex position and color vertex information.

### Claim 21 (depends on 17)

The system of claim 17 wherein triangle coding utilizes a triangle patch data unit.

### Claim 22 (depends on 21)

The system of claim 21 wherein the triangle patch data unit stores a triangle as a separate triangle, a triangle strip or a triangle fan.

### Claim 23 (depends on 17)

The system of claim 17 wherein triangle coding further includes a color expansion implementation which writes a color of a triangle as c0 c1 c2; cm cm cm; c0 c1 c2, where cm is a color value of a centroid position of the triangle.

### Claim 24 (depends on 23)

The system of claim 23 wherein triangle coding further includes color compression.