# US 11,910,054 B2 · Method and apparatus for decoding a 3D video

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,910,054 B2
- Document ID: US11910054B2
- Status: Granted
- Kind: B2
- Application number: 17/780,241
- Filed: 2020-11-24
- Published: 2023-01-05
- Granted: 2024-02-20
- Earliest claimed: 2019-11-29
- Assignees: InterDigital CE Patent Holdings, SAS
- Applicants: InterDigital CE Patent Holdings
- Inventors: Serge Travert; Jean Le Roux; Charles Salmon-Legagneur; Remi Houdaille
- Web page: https://patentgrep.com/patent/US11910054B2

## CPC

- H04N 13/161: Encoding, multiplexing or demultiplexing different image signal components (for multi-view video sequence encoding H04N19/597)
- H04N 21/44: Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N 19/597: specially adapted for multi-view video sequence encoding
- H04N 21/21805: {enabling multiple viewpoints, e.g. using a plurality of cameras}
- H04N 21/44209: {Monitoring of downstream path of the transmission network originating from a server, e.g. bandwidth variations of a wireless network (arrangements for maintenance or administration in data switching networks involving bandwidth and capacity management H04L41/0896)}
- H04N 21/44218: {Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV programme (methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45)}
- H04N 21/4621: {Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen (arrangements for using the results of monitoring on user's side in broadcast systems H04H60/65; flow control in packet networks H04L47/10)}
- H04N 21/816: {involving special video data, e.g 3D video}
- H04N 21/85406: {involving a specific file format, e.g. MP4 format}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method and an apparatus for decoding a volumetric video are disclosed. Such a method comprises receiving a data stream representative of a file comprising information for selecting, according to a rendering viewpoint, at least one atlas comprising color and depth data patches associated with a viewpoint in said volumetric video, said color and depth data patches being generated with respect to depth and color reference data acquired from a reference viewpoint in said volumetric video.

## Claims

### Claim 1

A method for decoding a volumetric video, the method comprising:
receiving a data stream representative of information for selecting, according to a rendering viewpoint, at least one atlas comprising color and depth data patches associated with a viewpoint in said volumetric video, said color and depth data patches being generated with respect to depth and color reference data acquired from a reference viewpoint in said volumetric video, wherein said information comprises a list of adaptive parallax parameters comprising:
a list of reference viewpoints comprising at least one reference viewpoint, and for each of said reference viewpoints:
associated coordinates of said at least one reference viewpoint in a volumetric cartesian coordinates system, and
at least one uniform resource identifier allowing to download an encoded data stream representative of an atlas associated with said reference viewpoint;
for each of said reference viewpoints in the list, a number of virtual cameras, each of said virtual cameras corresponding to a viewpoint in said volumetric video; and
for each of said virtual cameras:
associated coordinates of said viewpoint corresponding to said virtual camera in said 3D cartesian coordinates system, and
at least one uniform resource identifier allowing to download an encoded data stream representative of an atlas associated with said virtual camera.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
requesting from a server, using said information:
at least one data stream representative of a reference view comprising said color and depth reference data, and
at least one data stream representative of at least one selected atlas; and
decoding said at least one data stream representative of said reference view and said at least one data stream representative of said at least one selected atlas.

### Claim 3 (depends on 1)

The method of claim 1, wherein selecting the at least one atlas according to a rendering viewpoint of said volumetric video, comprises:
determining a network bandwidth available for requesting a next temporal segment of said volumetric video; and
determining, according to said information and according to said network bandwidth, a reference view and the at least one atlas associated with a viewpoint corresponding to a virtual camera for rendering said next temporal segment of said volumetric video.

### Claim 4 (depends on 1)

The method of claim 1, wherein selecting the at least one atlas according to a rendering viewpoint of said volumetric video uses at least one of:
predicting at least one future rendering viewpoint; and
determining regions of interest for which parallax is improved with a transmission of a corresponding atlas.

### Claim 5 (depends on 1)

The method of claim 1, wherein said adaptive parallax parameters further comprise at least one parameter belonging to the group:
a parameter quantifying a size of said encoded data stream representative of an atlas associated with a virtual camera or a reference viewpoint;
a group of parameters defining a projection associated with a virtual camera to be used for 2D to 3D conversion; and
a list of at least one zone of interest for which parallax is improved with a transmission of a corresponding atlas.

### Claim 6 (depends on 1)

The method of claim 1, wherein said information further comprises for at least one of said adaptive parallax parameters, an indication of a time when said at least one adaptive parallax parameter may change during rendering of said volumetric video.

### Claim 7 (depends on 6)

The method of claim 6, wherein said at least one adaptative parallax parameter which may change corresponds to the number of virtual cameras associated to a reference viewpoint.

### Claim 8 (depends on 1)

The method of claim 1, wherein said adaptative parallax parameters comprises, for at least one of said virtual cameras and/or one of said at least reference viewpoint, a plurality of uniform resource identifiers, each of said uniform resource identifiers being associated to a bit-rate level.

### Claim 9 (depends on 1)

The method of claim 1, further comprising:
receiving a data stream representative of updating the information, wherein said updating the information comprises at least one value for updating at least one of said adaptative parallax parameters.

### Claim 10

A method for encoding a volumetric video, comprising:
encoding at least one data stream representative of at least one reference view comprising color and depth reference data acquired from a reference viewpoint in the volumetric video;
encoding at least one data stream representative of at least one atlas comprising color and depth data patches acquired from a viewpoint in said volumetric video, said color and depth data patches being generated with respect to depth and color reference data; and
encoding a data stream comprising information for selecting the at least one atlas, according to a rendering viewpoint in said volumetric video, wherein said information comprises a list of adaptive parallax parameters comprising:
a list of reference viewpoints comprising at least one reference viewpoint, and for each of said reference viewpoints:
associated coordinates of said at least one reference viewpoint in a volumetric cartesian coordinates system, and
at least one uniform resource identifier allowing to download an encoded data stream representative of an atlas associated with said reference viewpoint;
for each of said reference viewpoints in the list, a number of virtual cameras, each of said virtual cameras corresponding to a viewpoint in said volumetric video; and
for each of said virtual cameras:
associated coordinates of said viewpoint corresponding to said virtual camera in said 3D cartesian coordinates system, and
at least one uniform resource identifier allowing to download an encoded data stream representative of an atlas associated with said virtual camera.

### Claim 11 (depends on 10)

The method of claim 10, wherein said adaptive parallax parameters further comprise at least one parameter belonging to the group:
a parameter quantifying a size of said encoded data stream representative of an atlas associated with a virtual camera or a reference viewpoint;
a group of parameters defining a projection associated with a virtual camera to be used for 2D to 3D conversion; and
a list of at least one zone of interest for which parallax is improved with a transmission of a corresponding atlas.

### Claim 12 (depends on 10)

The method of claim 10, wherein said information further comprises for at least one of said adaptive parallax parameters, an indication of a time when said at least one adaptive parallax parameter may change during rendering of said volumetric video.

### Claim 13 (depends on 12)

The method of claim 12, wherein said at least one adaptative parallax parameter which may change corresponds to the number of virtual cameras associated to a reference viewpoint.

### Claim 14 (depends on 10)

The method of claim 10, wherein said adaptative parallax parameters comprises, for at least one of said virtual cameras and/or one of said at least reference viewpoint, a plurality of uniform resource identifiers, each of said uniform resource identifiers being associated to a bit-rate level.

### Claim 15 (depends on 10)

The method of claim 10, further comprising receiving a data stream representative of updating information, wherein said updating information comprises at least one value for updating at least one of said adaptative parallax parameters.

### Claim 16

A non-transitory storage medium storing information for selecting, according to rendering viewpoint of a volumetric video, at least one atlas comprising color and depth data patches acquired from a viewpoint in said volumetric video, said color and depth data patches being generated with respect to depth and color reference data acquired from a reference viewpoint in said volumetric video, wherein said information comprises a list of adaptive parallax parameters comprising:
a list of reference viewpoints comprising at least one reference viewpoint, and for each of said reference viewpoints:
associated coordinates of said at least one reference viewpoint in a volumetric cartesian coordinates system, and
at least one uniform resource identifier allowing to download an encoded data stream representative of an atlas associated with said reference viewpoint;
for each of said reference viewpoints in the list, a number of virtual cameras, each of said virtual cameras corresponding to a viewpoint in said volumetric video; and
for each of said virtual cameras:
associated coordinates of said viewpoint corresponding to said virtual camera in said 3D cartesian coordinates system, and
at least one uniform resource identifier allowing to download an encoded data stream representative of an atlas associated with said virtual camera.

### Claim 17 (depends on 16)

The non-transitory storage medium of claim 16, wherein said adaptive parallax parameters further comprise at least one parameter belonging to the group:
a parameter quantifying a size of said encoded data stream representative of an atlas associated with a virtual camera or a reference viewpoint;
a group of parameters defining a projection associated with a virtual camera to be used for 2D to 3D conversion; and
a list of at least one zone of interest for which parallax is improved with a transmission of a corresponding atlas.

### Claim 18 (depends on 16)

The non-transitory storage medium of claim 16, wherein said information further comprises for at least one of said adaptive parallax parameters, an indication of a time when said at least one adaptive parallax parameter may change during rendering of said volumetric video.

### Claim 19 (depends on 18)

The non-transitory storage medium of claim 18, wherein said at least one adaptative parallax parameter which may change corresponds to the number of virtual cameras associated to a reference viewpoint.

### Claim 20 (depends on 16)

The non-transitory storage medium of claim 16, wherein said adaptative parallax parameters comprises, for at least one of said virtual cameras and/or one of said at least reference viewpoint, a plurality of uniform resource identifiers, each of said uniform resource identifiers being associated to a bit-rate level.

### Claim 21

An apparatus for decoding a volumetric video, the apparatus comprising a processor configured for:
receiving a data stream comprising information for selecting, according to a rendering viewpoint, at least one atlas comprising color and depth data patches acquired from a viewpoint in said volumetric video, said color and depth data patches being generated with respect to depth and color reference data acquired from a reference viewpoint in said volumetric video, wherein said information comprises a list of adaptive parallax parameters comprising:
a list of reference viewpoints comprising at least one reference viewpoint, and for each of said reference viewpoints:
associated coordinates of said at least one reference viewpoint in a volumetric cartesian coordinates system, and
at least one uniform resource identifier allowing to download an encoded data stream representative of an atlas associated with said reference viewpoint;
for each of said reference viewpoints in the list, a number of virtual cameras, each of said virtual cameras corresponding to a viewpoint in said volumetric video; and
for each of said virtual cameras:
associated coordinates of said viewpoint corresponding to said virtual camera in said 3D cartesian coordinates system, and
at least one uniform resource identifier allowing to download an encoded data stream representative of an atlas associated with said virtual camera.

### Claim 22 (depends on 21)

The apparatus of claim 21, further comprising:
means for requesting from a server, using said information:
at least one data stream representative of a reference view comprising said color and depth reference data, and
at least one data stream representative of the at least one selected atlas; and
means for decoding said at least one data stream representative of said reference view and said at least one data stream representative of said at least one selected atlas.

### Claim 23

An apparatus for encoding a volumetric video, the apparatus comprising a processor configured for:
encoding at least one data stream representative of at least one reference view comprising color and depth reference data acquired from a reference viewpoint in said volumetric video;
encoding at least one data stream representative of at least one atlas comprising color and depth data patches acquired from a viewpoint in said volumetric video, said color and depth data patches being generated with respect to depth and color reference data acquired from said reference viewpoint in said volumetric video; and
encoding a data stream comprising information for selecting at least one atlas, according to a rendering viewpoint of said volumetric video, wherein said information comprises a list of adaptive parallax parameters comprising:
a list of reference viewpoints comprising at least one reference viewpoint, and for each of said reference viewpoints:
associated coordinates of said at least one reference viewpoint in a volumetric cartesian coordinates system, and
at least one uniform resource identifier allowing to download an encoded data stream representative of an atlas associated with said reference viewpoint;
for each of said reference viewpoints in the list, a number of virtual cameras, each of said virtual cameras corresponding to a viewpoint in said volumetric video; and
for each of said virtual cameras:
associated coordinates of said viewpoint corresponding to said virtual camera in said 3D cartesian coordinates system, and
at least one uniform resource identifier allowing to download an encoded data stream representative of an atlas associated with said virtual camera.