# US 2023/0068178 A1 · A method, an apparatus and a computer program product for volumetric video encoding and decoding

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0068178 A1
- Document ID: US20230068178A1
- Status: Published application
- Kind: A1
- Application number: 17/790,370
- Filed: 2020-12-07
- Published: 2023-03-02
- Earliest claimed: 2020-01-02
- Assignees: Nokia Technologies Oy
- Applicants: Nokia Technologies Oy
- Inventors: Sebastian SCHWARZ; Kimmo ROIMELA; Lauri ILOLA; Lukasz KONDRAD
- Web page: https://patentgrep.com/patent/US20230068178A1

## CPC

- G06T 9/001: {Model-based coding, e.g. wire frame}
- H04N 13/239: using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular distance (H04N13/243 takes precedence)
- G06T 17/20: Finite element generation, e.g. wire-frame surface description, {tesselation}
- H04N 13/388: Volumetric displays, i.e. systems where the image is built up from picture elements distributed through a volume
- H04N 19/186: the unit being a colour or a chrominance component
- H04N 19/597: specially adapted for multi-view video sequence encoding
- H04N 19/70: characterised by syntax aspects related to video coding, e.g. related to compression standards
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method and technical equipment for encoding, where the method comprises at least receiving a video presentation frame, where the video presentation represents a three-dimensional data in the form of mesh data (810); separating from the mesh data information on vertices, wherein the information comprises at least connectivity data defining connections between vertices (820); determining parameters relating to said connectivity data (830); encoding the parameters to a first bitstream as a video component (840); and storing the encoded first bitstream for transmission to a rendering apparatus (850). In addition to encoding, also decoding is disclosed.

## Claims

### Claim at position 1

1-19. (canceled)

### Claim 20

A method, comprising:
receiving a video presentation frame, wherein the video presentation represents a three-dimensional data in the form of mesh data;
separating from the mesh data information on vertices, wherein the information comprises at least connectivity data defining connections between vertices;
determining parameters relating to said connectivity data;
encoding the parameters into a first bitstream as a video component; and
storing the encoded first bitstream for transmission to a rendering apparatus.

### Claim 21 (depends on 20)

The method according to claim 20, further comprising generating a mesh connectivity map by using a mesh connectivity table; wherein the mesh connectivity map signals per vertex indices of connected neighboring vertices; and packs mesh connectivity information per vertex in a single byte.

### Claim 22 (depends on 20)

The method according to claim 20, wherein the connectivity data comprises number of edges per vertex as a value in a luma channel.

### Claim 23 (depends on 20)

The method according to claim 20, further comprising encapsulating the connectivity data in an unused chroma channel of a geometry video.

### Claim 24 (depends on 20)

The method according to claim 20, wherein the connectivity data is embedded in an occupancy map.

### Claim 25 (depends on 20)

The method according to claim 20, wherein the information on vertices further comprises information on vertex coordinate and attributes, which are encoded to a second bitstream.

### Claim 26 (depends on 25)

The method according to claim 25, further comprising creating a final encoded bitstream from the first and the second bitstreams.

### Claim 27

A method, comprising:
receiving a compressed bitstream relating to a video presentation;
decoding, from the received bitstream, parameters relating to a connectivity data, wherein the connectivity data defines connections between vertices; and
using the connectivity data to reconstruct mesh data representing a three-dimensional data for the video presentation.

### Claim 28 (depends on 27)

The method according to claim 27, further comprising predicting mesh edges based on the connectivity data, wherein the predicting comprises:
decoding vertex locations;
reconstructing the vertex locations into a three-dimensional mesh frame;
comparing the reconstructed three-dimensional mesh frame to an original input mesh frame to generate a vertex correspondence; and
forming a mesh edge residual map based on the vertex correspondence.

### Claim 29

An apparatus comprising at least one processor, and a memory including a computer program code, wherein the memory and the computer program code are configured to, with the at least one processor, cause the apparatus to:
receive a video presentation frame, wherein the video presentation represents a three-dimensional data in the form of mesh data;
separate from the mesh data information on vertices, wherein the information comprises at least connectivity data defining connections between vertices;
determine parameters relating to said connectivity data;
encode the parameters into a first bitstream as a video component; and
store the encoded first bitstream for transmission to a rendering apparatus.

### Claim 30 (depends on 29)

The apparatus according to claim 29, wherein the apparatus is further caused to generate a mesh connectivity map by using a mesh connectivity table; wherein the mesh connectivity map signals per vertex indices of connected neighboring vertices; and packs mesh connectivity information per vertex in a single byte.

### Claim 31 (depends on 29)

The apparatus according to claim 29, wherein the connectivity data comprises number of edges per vertex as a value in a luma channel.

### Claim 32 (depends on 29)

The apparatus according to claim 29, wherein the apparatus is further caused to:
encapsulate the connectivity data in an unused chroma channel of a geometry video.

### Claim 33 (depends on 29)

The apparatus according to claim 29, wherein the connectivity data is embedded in an occupancy map.

### Claim 34 (depends on 29)

The apparatus according to claim 29, wherein the information on vertices comprises also information on vertex coordinate and attributes, which are encoded to a second bitstream.

### Claim 35 (depends on 34)

The apparatus according to claim 34, where in the apparatus is further caused to create a final encoded bitstream from the first and the second bitstreams.

### Claim 36

An apparatus comprising at least one processor, and a memory including a computer program code, wherein the memory and the computer program code are configured to, with the at least one processor, cause the apparatus to
receive a compressed bitstream relating to a video presentation;
decode from the received bitstream parameters relating to a connectivity data; wherein the connectivity data defines connections between vertices; and
use the connectivity data to reconstruct mesh data representing a three-dimensional data for a video presentation.

### Claim 37 (depends on 36)

The apparatus according to claim 36, wherein the apparatus is further caused to:
predict mesh edges based on the connectivity data, wherein the predicting comprises
decode vertex locations;
reconstruct the vertex locations into a three-dimensional mesh frame;
compare the reconstructed three-dimensional mesh frame to an original input mesh frame to generate a vertex correspondence; and
form a mesh edge residual map based on the vertex correspondence.
