Back to results
US 12,597,209 B2Granted
Using polygon mesh render composites during neural radiance field (NeRF) generation
AssigneeSony Interactive Entertainment Inc.
InventorsJoseph Logan Olson, Nasir Mohammad Khalid
Life of this patent
Estimating the term…
Abstract
A text-to-3D machine learning (ML) model is trained by rendering a pre-existing 3D model along with an associated depth map using plural camera settings and rendering a neural radiance field (NeRF) along with an associated depth map using the plural camera settings used in rendering the 3D model. A composite is established from the depth maps, which is used to train the ML model.
Claims13
A method, comprising:
- rendering a pre-existing 3D model along with an associated first depth map using plural camera settings;
- rendering a first neural radiance field (NeRF) along with an associated second depth map using the plural camera settings used in rendering the 3D model;
- using at least the associated first and second depth maps, compositing the 3D model and the first NeRF to establish a composite;
- and
- using the composite for training a text-to-3D machine learning (ML) model to output a second NeRF in response to text input,
- wherein the plural camera settings used to render the first depth map associated with the 3D model are the same plural camera settings used to render the second depth map associated with the first NeRF.
3 more elements
A device comprising:
- at least one processor assembly configured to:
- render a 3D model;
- render a first neural radiance field (NeRF);
- combine the 3D model and the first NeRF to establish a composite using data associated with the 3D model and data associated with the NeRF;
- and
- use the composite for training a text-to-3D machine learning (ML) model,
- wherein the data associated with the 3D model comprises a first depth map and the data associated with the NeRF comprises a second depth map, and
- wherein plural camera settings used to generate the first depth map associated with the 3D model are the same plural camera settings used to establish the second depth map associated with the NeRF.
5 more elements
A device comprising:
- at least one computer memory that is not a transitory signal and that comprises instructions executable by at least one processor assembly to:
- identify a 3D model rendering;
- identify a rendering of a NeRF;
- combine the renderings to render a composite using data associated with the 3D model and data associated with the NeRF;
- and
- use the composite to train at least one machine learning (ML) model to generate a NeRF based on text input,
- wherein the data associated with the 3D model comprises a first depth map and the data associated with the NeRF comprises a second depth map, and
- wherein plural camera settings used to generate the first depth map associated with the 3D model are the same plural camera settings used to establish the second depth map associated with the NeRF.
5 more elements
Dependencies are inferred from the claim text.