Mesh optimization using novel segmentation
AssigneeTencent America LLC
InventorsEthan Schur, Xiaozhong Xu, Shan Liu, Xiang Zhang
Life of this patent
Estimating the term…
Abstract
A method and apparatus that renders a mesh from one or more virtual camera views of an object, the one or more virtual camera views being projections of the mesh onto a plurality of 2D planes; inputs the one or more virtual camera views into a fully-convolutional model for real-time segmentation; generates one or more 2D pixel-wise masks based on the fully-convolutional model, the 2D pixel-wise masks being associated with one or more segmented elements; performs remeshing and mesh optimization using the one or more 2D pixel-wise masks; and generates an output mesh based on the remeshing and mesh optimization.
Claims18
A method performed by at least one processor, the method comprising:
- rendering a mesh from one or more virtual camera views of an object, the one or more virtual camera views being projections of the mesh onto a plurality of two-dimensional (2D) planes;
- inputting the one or more virtual camera views into a fully-convolutional model for real-time segmentation;
- generating one or more 2D pixel-wise masks based on the fully-convolutional model, the 2D pixel-wise masks being associated with one or more segmented elements, the one or more 2D pixel-wise masks are used as inputs to a scene graph processor for streaming a plurality of scene graph elements to a plurality of decoders, intermediary software, and renderers;
- performing remeshing and mesh optimization using the one or more 2D pixel-wise masks;
- and
- generating an output mesh based on the remeshing and mesh optimization.
An apparatus comprising:
- at least one memory configured to store program code;
- and
- at least one processor configured to read the program code and operate as instructed by the program code, the program code comprising:
- rendering code configured to cause the at least one processor to render a mesh from one or more virtual camera views of an object, the one or more virtual camera views being projections of the mesh onto a plurality of two-dimensional (2D) planes;
- inputting code configured to cause the at least one processor to input the one or more virtual camera views into a fully-convolutional model for real-time segmentation;
- first generating code configured to cause the at least one processor to generate one or more 2D pixel-wise masks based on the fully-convolutional model, the 2D pixel-wise masks being associated with one or more segmented elements, the one or more 2D pixel-wise masks are used as inputs to a scene graph processor for streaming a plurality of scene graph elements to a plurality of decoders, intermediary software, and renderers;
- remeshing code configured to cause the at least one processor to perform remeshing and mesh optimization using the one or more 2D pixel-wise masks;
- and
- second generating code configured to cause the at least one processor to generate an output mesh based on the remeshing and mesh optimization.
A non-transitory computer-readable storage medium, storing instructions, which, when executed by at least one processor, cause the at least one processor to:
- render a mesh from one or more virtual camera views of an object, the one or more virtual camera views being projections of the mesh onto a plurality of two-dimensional (2D) planes;
- input the one or more virtual camera views into a fully-convolutional model for real-time segmentation;
- generate one or more 2D pixel-wise masks based on the fully-convolutional model, the 2D pixel-wise masks being associated with one or more segmented elements, the one or more 2D pixel-wise masks are used as inputs to a scene graph processor for streaming a plurality of scene graph elements to a plurality of decoders, intermediary software, and renderers;
- perform remeshing and mesh optimization using the one or more 2D pixel-wise masks;
- and
- generate an output mesh based on the remeshing and mesh optimization.
Dependencies are inferred from the claim text.