# US 2021/0241496 A1 · Method and apparatus for encoding and decoding volumetric video data > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0241496 A1 - Document ID: US20210241496A1 - Status: Published application - Kind: A1 - Application number: 17/051,820 - Filed: 2019-05-07 - Published: 2021-08-05 - Earliest claimed: 2018-05-09 - Assignees: Nokia Technologies Oy - Applicants: Nokia Technologies Oy - Inventors: Mika PESONEN; Sebastian SCHWARZ; Vida FAKOUR SEVOM - Web page: https://patentgrep.com/patent/US20210241496A1 ## CPC - H04N 19/597: specially adapted for multi-view video sequence encoding - G06T 9/001: {Model-based coding, e.g. wire frame} - H04N 19/137: Motion inside a coding unit, e.g. average field, frame or block difference - H04N 19/176: the region being a block, e.g. a macroblock - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A method, apparatus and computer program product provide for encoding and decoding volumetric video data. In relation to encoding, one or more patches into which a point cloud representative of volumetric video data has been decomposed are evaluated by identifying one or more areas within a respective patch in which a difference in depth values between corresponding pixels of at least two depth planes satisfy a predefined threshold. For an area within the respective patch that has been identified, a sub patch is generated representative of the depth values of one of the depth planes. The patches are mapped including the one or more patches and the sub patch to a grid and an image is generated based upon the patches mapped to the grid. ## Claims ### Claim at position 1 1-27. (canceled) ### Claim 28 A method comprising: evaluating one or more patches into which a point cloud representative of volumetric video data has been decomposed, wherein evaluating the one or more patches comprises identifying one or more areas within a respective patch in which a difference in depth values between corresponding pixels of at least two depth planes satisfies a predefined threshold; for an area within the respective patch that has been identified, generating a sub patch representative of the depth values of one of the depth planes; mapping patches including the one or more patches and the sub patch to a grid; and generating an image based upon the patches mapped to the grid. ### Claim 29 (depends on 28) The method according to claim 28, wherein the grid is comprised of a plurality of blocks, and wherein evaluating the one or more patches comprises separately evaluating one or more of the plurality of blocks of the respective patch to identify one or more areas within the respective patch corresponding to the respective blocks in which the difference in depth values between corresponding pixels of at least two depth planes satisfy the predefined threshold. ### Claim 30 (depends on 28) The method according to claim 28, wherein identifying the one or more areas within the respective patch comprises: identifying one or more pixels within the respective patch for which the difference in the depth values of the at least two depth planes satisfies the predefined threshold; and filling one or more holes between the one or more pixels that were identified to create the one or more areas within the respective patch. ### Claim 31 (depends on 30) The method according to claim 30, wherein identifying the one or more areas within the respective patch further comprises filtering the one or more pixels that were identified to eliminate single pixels that are not surrounded by other pixels prior to filling one or more holes. ### Claim 32 (depends on 30) The method according to claim 30, wherein identifying the one or more areas within the respective patch further comprises thresholding the one or more pixels after filling the one or more holes such that the pixels have one of two values. ### Claim 33 (depends on 28) The method according to claim 28 further comprising associating metadata with the sub patch including parent patch information that identifies the respective patch from which the sub patch was generated. ### Claim 34 (depends on 28) The method according to claim 28 further comprising associating metadata with the sub patch including sub patch location information that identifies a location of the sub patch in relation to the respective patch from which the sub patch was generated. ### Claim 35 (depends on 28) The method according to claim 28 further comprising associating metadata with the respective patch including children information that identifies one or more sub patches that are generated from the respective patch. ### Claim 36 An apparatus comprising at least one processor and at least one memory including computer program code for one or more programs, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to: evaluate one or more patches into which a point cloud representative of volumetric video data has been decomposed by identifying one or more areas within a respective patch in which a difference in depth values between corresponding pixels of at least two depth planes satisfy a predefined threshold; for an area within the respective patch that has been identified, generate a sub patch representative of the depth values of one of the depth planes; map patches including the one or more patches and the sub patch to a grid; and generate an image based upon the patches mapped to the grid. ### Claim 37 (depends on 36) The apparatus according to claim 36, wherein the grid is comprised of a plurality of blocks, and wherein to evaluate the one or more patches, the apparatus is further caused to separately evaluate one or more of the plurality of blocks of the respective patch to identify one or more areas within the respective patch corresponding to the respective blocks in which the difference in depth values between corresponding pixels of at least two depth planes satisfy the predefined threshold. ### Claim 38 (depends on 36) The apparatus according to claim 36, wherein to identify one or more areas within the respective patch, the apparatus is further caused to: identify one or more pixels within the respective patch for which the difference in the depth values of the at least two depth planes satisfies the predefined threshold; and fill one or more holes between the one or more pixels that were identified to create the one or more areas within the respective patch. ### Claim 39 (depends on 38) The apparatus according to claim 38, wherein to identify one or more areas within the respective patch, the apparatus is further caused to filter the one or more pixels that were identified to eliminate single pixels that are not surrounded by other pixels prior to filling one or more holes. ### Claim 40 (depends on 38) The apparatus according to claim 38, wherein to identify one or more areas within the respective patch, the apparatus is further caused to, threshold the one or more pixels after filling one or more holes such that the pixels have one of two values. ### Claim 41 (depends on 36) The apparatus according to claim 36, wherein the at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus to associate metadata with the sub patch including parent patch information that identifies the respective patch from which the sub patch was generated. ### Claim 42 (depends on 36) The apparatus according to claim 36, wherein the at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus to associate metadata with the sub patch including sub patch location information that identifies a location of the sub patch in relation to the respective patch from which the sub patch was generated. ### Claim 43 (depends on 36) The apparatus according to claim 36 wherein the at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus to associate metadata with the respective patch including children information that identifies one or more sub patches that are generated from the respective patch. ### Claim 44 A method comprising: decoding one or more patches that fail to include parent patch information identifying a respective patch from which the one or more patches were generated; for each of a plurality of pixels of a sub patch that includes parent patch information and sub patch location information that identifies a location of the sub patch in relation to the respective patch from which the sub patch was generated, comparing depth values of the pixel of the sub patch and a corresponding pixel of the patch from which the sub patch was generated to determine a difference therebetween; in an instance in which the difference satisfies a predefined threshold indicative of a variation therebetween, decoding the pixel of the sub patch; and in an instance in which the difference fails to satisfy the predefined threshold, discarding the pixel of the sub patch without decoding. ### Claim 45 An apparatus comprising at least one processor and at least one memory including computer program code for one or more programs, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to: decode one or more patches that fail to include parent patch information identifying a respective patch from which the one or more patches were generated; for each of a plurality of pixels of a sub patch that includes parent patch information and sub patch location information that identifies a location of the sub patch in relation to the respective patch from which the sub patch was generated, compare depth values of the pixel of the sub patch and a corresponding pixel of the patch from which the sub patch was generated to determine a difference therebetween; in an instance in which the difference satisfies a predefined threshold indicative of a variation therebetween, decode the pixel of the sub patch; and in an instance in which the difference fails to satisfy the predefined threshold, discard the pixel of the sub patch without decoding.