# US 12,120,326 B2 · Method and apparatus for encoding and decoding 3D video content > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,120,326 B2 - Document ID: US12120326B2 - Status: Granted - Kind: B2 - Application number: 13/138,459 - Filed: 2010-02-19 - Published: 2011-12-08 - Granted: 2024-10-15 - Earliest claimed: 2009-02-19 - Assignees: InterDigital Madison Patent Holdings, SAS - Applicants: Dong Tian; Po-Lin Lai; Jiancong Luo - Inventors: Dong Tian; Po-Lin Lai; Jiancong Luo - Web page: https://patentgrep.com/patent/US12120326B2 ## CPC - H04N 19/30: using hierarchical techniques, e.g. scalability (H04N19/63 takes precedence) - H04N 13/128: Adjusting depth or disparity - H04N 13/194: Transmission of image signals - H04N 19/46: Embedding additional information in the video signal during the compression process (H04N19/517, H04N19/68, H04N19/70 take precedence) - H04N 19/597: specially adapted for multi-view video sequence encoding - H04N 19/70: characterised by syntax aspects related to video coding, e.g. related to compression standards - H04N 2213/003: Aspects relating to the "2D+depth" image format - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Several implementations relate to 3D video formats. One or more implementations provide adaptations to MVC and SVC to allow 3D video formats to be used. According to a general aspect, a set of images including video and depth is encoded. The set of images is related according to a particular 3D video format, and are encoded in a manner that exploits redundancy between the set of images. The encoded images are arranged in a bitstream in a particular order, based on the particular 3D video format that relates to the images. The particular order is indicated in the bitstream using signaling information. According to another general aspect, a bitstream is accessed that includes the encoded set of images. The signaling information is also accessed. The set of images is decoded using the signaling information. ## Claims ### Claim 1 A method comprising: encoding at least one set of 3D video components of a 3D video format in a layer-based structured bitstream, wherein each of the 3D video components is assigned to one layer of the layer-based structured bitstream, at least one layer of the layer-based structured bitstream being coded using one or more other layers as a reference; arranging the 3D video components of the at least one set of 3D video components in the layer-based structured bitstream in an order; and encoding of signaling information of the layer-based structured bitstream, the signaling information indicating the order, wherein the signaling information comprises a first syntax element indicating the 3D video format among at least: a first 3D video format wherein the 3D video components of the first 3D video format comprise the 2D video and corresponding depth components, and a second 3D video format wherein the 3D video components of the second 3D video format comprise the 2D video, corresponding depth components, 2D occlusion video component and corresponding depth occlusion video component, wherein the 3D video components of the at least one set of 3D video components are arranged in the layer-based structured bitstream: in a first order when the 3D video components of the set are in the first 3D video format, and in a second order when the 3D video components of the set are in the second 3D video format. ### Claim 2 (depends on 1) The method of claim 1, wherein the signaling information further comprises: a second syntax element indicating whether a first 3D video component of the set of 3D video components is present in the layer-based structured bitstream; and a third syntax element indicating a first layer in the bitstream which corresponds to the first 3D video component of a set of 3D video components which is present in the layer-based structured bitstream. ### Claim 3 (depends on 2) The method of claim 2, wherein the first 3D video component is a 2D video component of the set of 3D video components, a depth component corresponding to a 2D video component of the set of 3D video components, a 2D occlusion video component of the set of 2D video components or a depth occlusion video component corresponding to an occlusion video component of the set of 3D video components. ### Claim 4 (depends on 2) The method of claim 2, wherein the signaling information is included in an SEI message and the signaling information further comprises a fourth syntax element indicating whether the persistence of the indications provided by the first, second and/or third syntax elements in any previous SEI message in output order is canceled. ### Claim 5 (depends on 1) The method of claim 1, wherein the layer-based structured bitstream is compliant with a scalable video coding extension of a video coding standard. ### Claim 6 An apparatus comprising one or more processors configured to perform at least: encoding at least one set of 3D video components of a 3D video format in a layer-based structured bitstream, wherein each of the 3D video components is assigned to one layer of the layer-based structured bitstream, at least one layer of the layer-based structured bitstream being coded using one or more other layers as a reference; arranging the 3D video components of the at least one set of 3D video components in the layer-based structured bitstream in an order; and encoding of signaling information of the layer-based structured bitstream, the signaling information indicating the order, wherein the signaling information comprises a first syntax element indicating the 3D video format among at least: a first 3D video format wherein the 3D video components of the first 3D video format comprise the 2D video and corresponding depth components, and a second 3D video format wherein the 3D video components of the second 3D video format comprise the 2D video, corresponding depth components, 2D occlusion video component and corresponding depth occlusion video component, wherein the 3D video components of the at least one set of 3D video components are arranged in the layer-based structured bitstream: in a first order when the 3D video components of the set are in the first 3D video format, and in a second order when the 3D video components of the set are in the second 3D video format. ### Claim 7 (depends on 6) The apparatus of claim 6, wherein the signaling information further comprises: a second syntax element indicating whether a first 3D video component of the set of 3D video components is present in the layer-based structured bitstream; and a third syntax element indicating a first layer in the bitstream which corresponds to the first 3D video component of a set of 3D video components which is present in the layer-based structured bitstream. ### Claim 8 (depends on 7) The apparatus of claim 7, wherein the signaling information is included in an SEI message and the signaling information further comprises a fourth syntax element indicating whether the persistence of the indications provided by the first, second and/or third syntax elements in any previous SEI message in output order is canceled. ### Claim 9 (depends on 6) The apparatus of claim 6, wherein the first 3D video component is a 2D video component of the set of 3D video components, a depth component corresponding to a 2D video component of the set of 3D video components, a 2D occlusion video component of the set of 2D video components or a depth occlusion video component corresponding to an occlusion video component of the set of 3D video components. ### Claim 10 (depends on 6) The apparatus of claim 6, wherein the layer-based structured bitstream is compliant with a scalable video coding extension of a video coding standard. ### Claim 11 A method comprising: accessing a layer-based structured bitstream, the layer-based structured bitstream including at least one set of 3D video components of a 3D video format, each of the 3D video components being assigned to one layer of the layer-based structured bitstream, at least one layer of the layer-based structured bitstream being coded using one or more other layers as a reference; accessing signaling information of the layer-based structured bitstream, the signaling information indicating an order in which the 3D video components of the at least one set of 3D video components are arranged in the layer-based structured bitstream, wherein the signaling information comprises a first syntax element indicating the 3D video format among at least: a first 3D video format wherein the 3D video components of the first 3D video format comprise the 2D video and corresponding depth components, and a second 3D video format wherein the 3D video components of the second 3D video format comprise the 2D video, corresponding depth components, 2D occlusion video component and corresponding depth occlusion video component, wherein the 3D video components of the at least one set of 3D video components are arranged in the layer-based structured bitstream: in a first order when the 3D video components of the set are in the first 3D video format, and in a second order when the 3D video components of the set are in the second 3D video format; and decoding the at least one set of 3D video components using the signaling information. ### Claim 12 (depends on 11) The method of claim 11, wherein the signaling information further comprises: a second syntax element indicating whether a first 3D video component of the set of 3D video components is present in the layer-based structured bitstream; and a third syntax element indicating a first layer in the layer-based structured bitstream which corresponds to the first 3D video component of the set of 3D video components which is present in the layer-based structured bitstream. ### Claim 13 (depends on 12) The method of claim 12, wherein the first 3D video component is a 2D video component of the set of 3D video components, a depth component corresponding to a 2D video component of the set of 3D video components, 2D occlusion video component of the set of 2D video components or a depth occlusion video component corresponding to an occlusion video component of the set of 3D video components. ### Claim 14 (depends on 12) The method of claim 12, wherein the signaling information is included in an SEI message and the signaling information further comprises a fourth syntax element indicating whether the persistence of the indications provided by the first, second and/or third syntax elements in any previous SEI message in output order is canceled. ### Claim 15 (depends on 11) The method of claim 11, wherein the signaling information is included in an SEI message. ### Claim 16 (depends on 11) The method of claim 11, wherein the layer-based structured bitstream is compliant with a scalable video coding extension of a video coding standard. ### Claim 17 An apparatus comprising one or more processors configured to perform at least: accessing a layer-based structured bitstream, the layer-based structured bitstream including at least one set of 3D video components of a 3D video format, each of the 3D video components being assigned to one layer of the layer-based structured bitstream, at least one layer of the layer- based structured bitstream being coded using one or more other layers as a reference; accessing signaling information of the layer-based structured bitstream, the signaling information indicating an order in which the 3D video components of the at least one set of 3D video components are arranged in the layer-based structured bitstream, wherein the signaling information comprises a first syntax element indicating the 3D video format among at least: a first 3D video format wherein the 3D video components of the first 3D video format comprise the 2D video and corresponding depth components, and a second 3D video format wherein the 3D video components of the second 3D video format comprise the 2D video, corresponding depth components, 2D occlusion video component and corresponding depth occlusion video component, wherein the 3D video components of the at least one set of 3D video components are arranged in the layer-based structured bitstream: in a first order when the 3D video components of the set are in the first 3D video format, and in a second order when the 3D video components of the set are in the second 3D video format; and decoding the at least one set of 3D video components using the signaling information. ### Claim 18 (depends on 17) The apparatus of claim 17, wherein the signaling information further comprises: a second syntax element indicating whether a first 3D video component of the set of 3D video components is present in the layer-based structured bitstream; and a third syntax element indicating a first layer in the layer-based structured bitstream which corresponds to the first 3D video component of a set of 3D video components which is present in the layer-based structured bitstream. ### Claim 19 (depends on 18) The apparatus of claim 18, wherein the first 3D video component is a 2D video component of the set of 3D video components, a depth component corresponding to a 2D video component of the set of 3D video components, 2D occlusion video component of the set of 2D video components or a depth occlusion video component corresponding to an occlusion video component of the set of 3D video components. ### Claim 20 (depends on 18) The apparatus of claim 18, wherein the signaling information is included in an SEI message and the signaling information further comprises a fourth syntax element indicating whether the persistence of the indications provided by the first, second and/or third syntax elements in any previous SEI message in output order is canceled. ### Claim 21 (depends on 18) The apparatus of claim 18, wherein the signaling information is included in an SEI message. ### Claim 22 (depends on 18) The apparatus of claim 18, wherein the layer-based structured bitstream is compliant with a scalable video coding extension of a video coding standard. ### Claim 23 A non-transitory processor-readable medium having stored thereon instructions operative, when executed by a processor, to cause the a processor to: access a layer-based structured bitstream, the layer-based structured bitstream including at least one set of 3D video components of a 3D video format, each of the 3D video components being assigned to one layer of the layer-based structured bitstream, at least one layer of the layer-based structured bitstream being coded using one or more other layers as a reference; access signaling information of the layer-based structured bitstream, the signaling information indicating an order in which the 3D video components of the at least one set of 3D video components are arranged in the bitstream, wherein the signaling information comprising a first syntax element indicating the 3D video format among at least: a first 3D video format wherein the 3D video components of the first 3D video format comprise the 2D video and corresponding depth components, and a second 3D video format wherein the 3D video components of the second 3D video format comprise the 2D video, corresponding depth components, 2D occlusion video component and corresponding depth occlusion video component, wherein the 3D video components of the at least one set of 3D video components are arranged in the layer-based structured bitstream: in a first order when the 3D video components of the set are in the first 3D video format, and in a second order when the 3D video components of the set are in the second 3D video format; and decode the at least one set of 3D video components using the signaling information. ### Claim 24 (depends on 23) The non-transitory processor-readable medium of claim 23, wherein the video signal further comprises: a second syntax element indicating whether a first 3D video component of the set of 3D video components is present in the layer-based structured bitstream; and a third syntax element indicating a first layer in the layer-based structured bitstream which corresponds to the first 3D video component of the set of 3D video components which is present in the layer-based structured bitstream. ### Claim 25 (depends on 24) The non-transitory processor-readable medium of claim 24, wherein the signaling information is included in an SEI message and the signaling information further comprises a fourth syntax element indicating whether the persistence of the indications provided by the first, second and/or third syntax elements in any previous SEI message in output order is canceled. ### Claim 26 (depends on 24) The non-transitory processor-readable medium of claim 24, wherein the first 3D video component is a 2D video of the set of 3D video components, a depth component corresponding to a 2D video of the set of 3D video components, 2D occlusion video component of the set of 2D video components or a depth occlusion video component corresponding to an occlusion video component of the set of 3D video components. ### Claim 27 A non-transitory processor-readable medium having stored thereon a video signal structure, the video signal comprising a signaling section including signaling information indicating an order in which 3D video components of at least one set of 3D video components of a 3D video format are arranged in a layer-based structured bitstream, wherein the signaling information comprises a first syntax element indicating the 3D video format among at least: a first 3D video format wherein the 3D video components of the first 3D video format comprise the 2D video and corresponding depth components, and a second 3D video format wherein the 3D video components of the second 3D video format comprise the 2D video, corresponding depth components, 2D occlusion video component and corresponding depth occlusion video component, wherein the 3D video components of the at least one set of 3D video components are arranged in the layer-based structured bitstream: in a first order when the 3D video components of the set are in the first 3D video format, and in a second order when the 3D video components of the set are in the second 3D video format, each of the 3D video components being assigned to one layer of the layer-based structured bitstream, one or more layers being capable of being used as a reference for at least one other layer to exploit redundancy there between. ### Claim 28 (depends on 27) The non-transitory processor-readable medium of claim 27, wherein the video signal further comprises: a second syntax element indicating whether a first 3D video component of the set of 3D video components is present in the layer-based structured bitstream; and a third syntax element indicating a first layer in the layer-based structured bitstream which corresponds to the first 3D video component of the set of 3D video components which is present in the layer-based structured bitstream. ### Claim 29 (depends on 28) The non-transitory processor-readable medium of claim 28, wherein the signaling information is included in an SEI message and the signaling information further comprises a fourth syntax element indicating whether the persistence of the indications provided by the first, second and/or third syntax elements in any previous SEI message in output order is canceled. ### Claim 30 (depends on 27) The non-transitory processor-readable medium of claim 27, wherein the first 3D video component is a 2D video of the set of 3D video components, a depth component corresponding to a 2D video of the set of 3D video components, 2D occlusion video component of the set of 2D video components or a depth occlusion video component corresponding to an occlusion video component of the set of 3D video components.