# US 12,120,306 B2 · V3C patch remeshing for dynamic mesh coding > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,120,306 B2 - Document ID: US12120306B2 - Status: Granted - Kind: B2 - Application number: 18/116,937 - Filed: 2023-03-03 - Published: 2023-09-21 - Granted: 2024-10-15 - Earliest claimed: 2022-03-18 - Assignees: Nokia Technologies Oy - Applicants: Nokia Technologies Oy - Inventors: Patrice Rondao Alface; Aleksei Martemianov; Lukasz Kondrad; Lauri Aleksi Ilola; Sebastian Schwarz - Web page: https://patentgrep.com/patent/US12120306B2 ## CPC - H04N 19/132: Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking - H04N 19/597: specially adapted for multi-view video sequence encoding - G06T 9/001: {Model-based coding, e.g. wire frame} - G06T 17/205: {Re-meshing} - H04N 19/182: the unit being a pixel - H04N 19/20: using video object coding - H04N 19/70: characterised by syntax aspects related to video coding, e.g. related to compression standards - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract An apparatus comprising circuitry configured to: obtain a three-dimensional model comprising at least one patch, at least one geometry component, at least one occupancy component, and zero or more texture components; subsample the at least one geometry component of the at least one patch of a three-dimensional object at occupied positions using a subsampling criterion; define respective search windows around the respective occupied positions; select respective salient points relative to the respective occupied positions within the respective search windows; triangulate the salient points to approximate a shape of the three-dimensional object; detect zero or more triangles that overlap with at least one unoccupied pixel; split the zero or more triangles that overlap with at least one unoccupied pixel until no triangle overlaps with the unoccupied pixels; and add zero or more additional triangles close to a border of the three-dimensional object to generate a resulting mesh signaled to a decoder. ## Claims ### Claim 1 An apparatus comprising: at least one processor; and at least one non-transitory memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to: obtain a three-dimensional model comprising at least one patch, at least one geometry component, at least one occupancy component, and zero or more texture components; subsample the at least one geometry component of the at least one patch of a three-dimensional object at occupied positions using a subsampling criterion; define respective search windows around the respective occupied positions; select respective salient points relative to the respective occupied positions within the respective search windows; triangulate the salient points to approximate a shape of the three-dimensional object; detect zero or more triangles that overlap with at least one unoccupied pixel; split the zero or more triangles that overlap with at least one unoccupied pixel until no triangle overlaps with the unoccupied pixels; and add zero or more additional triangles close to a border of the three-dimensional object to generate a resulting mesh that is signaled to a decoder. ### Claim 2 (depends on 1) The apparatus of claim 1, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: signal a subsampling factor for the at least one patch within a bitstream using a patch data unit to signal a first sampling rate of a first coordinate axis of the at least one geometry component and a second sampling rate of a second coordinate axis of the at least one geometry component. ### Claim 3 (depends on 1) The apparatus of claim 1, wherein the search windows are non-overlapping, such that at least one pixel or at least one column of pixels separates one search window from another search window. ### Claim 4 (depends on 1) The apparatus of claim 1, wherein the salient points are chosen based on at least one of: a local extremum; a singular point on a curvature of the three-dimensional object; an umbilical point; or a point on a seam of texture coordinates or edges of texture pixels of a texture component. ### Claim 5 (depends on 1) The apparatus of claim 1, wherein the zero or more triangles are split with sampling an occupied pixel substantially close to an edge of the zero or more triangles that overlap with at least one unoccupied pixel. ### Claim 6 (depends on 1) The apparatus of claim 1, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: add at least one additional triangle within a core of the at least one patch. ### Claim 7 (depends on 1) The apparatus of claim 1, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: smooth the resulting mesh. ### Claim 8 (depends on 1) The apparatus of claim 1, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: signal a local search window size to the decoder, wherein the local search window size is a size of the search windows surrounding a subsampled occupied pixel of the at least one geometry component; signal information related to the salient points to the decoder, wherein the salient points are chosen so that a preliminary mesh approximates the shape of the three-dimensional object; and signal information related to iterative triangle refinement of the preliminary mesh, the iterative triangle refinement performed with adding one of the occupied positions to cover a percentage of contour pixels of the at least one geometry component. ### Claim 9 An apparatus comprising: at least one processor; and at least one non-transitory memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to: receive a three-dimensional object comprising at least one patch, at least one geometry component, at least one occupancy component, and zero or more texture components; define multiple sets of parameters describing sub-sampling criteria, a size of a search window, and types of salient points; generate a mesh using a depth patch triangulation method using the sets of parameters to approximate a shape of the three-dimensional object; evaluate a quality of a depth patch triangulation compared to a reconstructed three-dimensional object reconstructed without triangulated depth patches; iterate until a reconstructed three-dimensional object using the depth patch triangulation method reaches an expected improved quality; derive at least one parameter used to code the mesh with improved quality; and transmit to a decoder the coded mesh as a bitstream and the at least one parameter. ### Claim 10 (depends on 9) The apparatus of claim 9, wherein the at least one parameter is transmitted within a patch data unit syntax structure. ### Claim 11 (depends on 9) The apparatus of any claim 9, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: signal a level of detail patch data unit syntax element to the decoder; wherein the level of detail patch data unit syntax element is configured to be used with the decoder to reconstruct a triangulation of the at least one patch. ### Claim 12 (depends on 11) The apparatus of claim 11, wherein, in response to the level of detail patch data unit syntax element evaluating as true, a first coordinate scale and a second coordinate scale indication are used to define a sampling of the mesh. ### Claim 13 (depends on 9) The apparatus of claim 9, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: signal a subsampling of the mesh within an atlas sequence parameter set or a supplemental enhancement information message. ### Claim 14 (depends on 9) The apparatus of claim 9, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: signal a local search window size to the decoder, wherein the local search window size is a size of a search window surrounding a subsampled occupied pixel of the at least one geometry component; signal information related to at least one salient point to the decoder, wherein the at least one salient point is chosen so that the mesh approximates the shape of the three-dimensional object; and signal information to the decoder related to iterative triangle refinement of the mesh, the iterative triangle refinement performed with adding at least one occupied position to cover a percentage of contour pixels of the at least one geometry component. ### Claim 15 An apparatus comprising: at least one processor; and at least one non-transitory memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to: receive a coded mesh as a bitstream, wherein the mesh is used to approximate a shape of a three-dimensional object; wherein the mesh has been coded with depth patch triangulation method using multiple sets of parameters to approximate the shape of the three-dimensional object; wherein the multiple sets of parameters describe sub-sampling criteria, a size of a search window, and types of salient points; receive at least one parameter used to code the mesh with improved quality; and reconstruct the mesh from the bitstream using the at least one parameter. ### Claim 16 (depends on 15) The apparatus of claim 15, wherein the at least one parameter is received within a patch data unit syntax structure. ### Claim 17 (depends on 15) The apparatus of claim 15, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: receive signaling comprising a level of detail patch data unit syntax element; and reconstruct a triangulation of the at least one patch based on the level of detail patch data unit syntax element. ### Claim 18 (depends on 17) The apparatus of claim 17, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: in response to the level of detail patch data unit syntax element evaluating as true, use a first coordinate scale and a second coordinate scale indication to define a sampling of the mesh. ### Claim 19 (depends on 15) The apparatus of claim 15, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: receive signaling of a subsampling of the mesh within an atlas sequence parameter set or a supplemental enhancement information message. ### Claim 20 (depends on 15) The apparatus of claim 15, wherein the instructions, when executed by the at least one processor, cause the apparatus at least to: receive signaling of a local search window size from an encoder, wherein the local search window size is a size of a search window surrounding a subsampled occupied pixel of at least one geometry component; receive signaling of information related to at least one salient point from the encoder, wherein the at least one salient point is chosen so that the mesh approximates the shape of the three-dimensional object; and receive signaling of information from the encoder related to iterative triangle refinement of the mesh, the iterative triangle refinement performed with adding at least one occupied position to cover a percentage of contour pixels of the at least one geometry component. ### Claim 21 An apparatus comprising: at least one processor; and at least one non-transitory memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to: receive signaling from an encoder, the signaling related to encoding a three-dimensional model comprising at least one patch, at least one geometry component, at least one occupancy component, and zero or more texture components; subsample the at least one geometry component of the at least one patch of a three-dimensional object at occupied positions using a subsampling criterion; define respective search windows around the respective occupied positions; select respective salient points relative to the respective occupied positions within the respective search windows; triangulate the salient points to approximate a shape of the three-dimensional object; detect zero or more triangles that overlap with at least one unoccupied pixel; split the zero or more triangles that overlap with at least one unoccupied pixel until no triangle overlaps with the unoccupied pixels; add zero or more additional triangles close to a border of the three-dimensional object; and reconstruct a mesh of the three-dimensional object, based on the received signaling. ### Claim 22 (depends on 21) The apparatus of claim 21, wherein the signaling comprises at least one of: mesh reconstruction information; a type at least one of the salient points; a local window size; or a percentage of contours. ### Claim 23 (depends on 21) The apparatus of claim 21, wherein the subsampling of the at least one geometry component of the at least one patch of the three-dimensional object at occupied positions using the subsampling criterion is performed using the received signaling. ### Claim 24 (depends on 23) The apparatus of claim 23, wherein the received signaling comprises the subsampling criterion. ### Claim 25 (depends on 21) The apparatus of claim 21, wherein the defining of the respective search windows around the respective occupied positions is performed using the received signaling. ### Claim 26 (depends on 25) The apparatus of claim 25, wherein the received signaling comprises a size of the respective search windows. ### Claim 27 (depends on 21) The apparatus of claim 21, wherein the selecting of the respective salient points relative to the respective occupied positions within the respective search windows is performed using the received signaling. ### Claim 28 (depends on 27) The apparatus of claim 27, wherein the received signaling comprises a type of the respective salient points. ### Claim 29 (depends on 21) The apparatus of claim 21, wherein the adding of the zero or more additional triangles close to the border of the three-dimensional object is performed using the received signaling. ### Claim 30 (depends on 29) The apparatus of claim 29, wherein the received signaling comprises a percentage of contours.