# US 12,718,461 B2 · Generating physically-based rendering (PBR) textures using image-based diffusion > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,718,461 B2 - Document ID: US12718461B2 - Status: Granted - Kind: B2 - Application number: 18/401,262 - Filed: 2023-12-29 - Published: 2025-07-03 - Granted: 2026-08-25 - Earliest claimed: 2023-12-29 - Assignees: Unity Technologies SF - Applicants: Unity Technologies SF - Inventors: Shimon Vainer; Simon Johannes Donné; Mark Benedikt Boss; Mathias Parger - Web page: https://patentgrep.com/patent/US12718461B2 ## CPC - G06T 15/04: Texture mapping - H04N 13/351: for displaying simultaneously - H04N 19/597: specially adapted for multi-view video sequence encoding - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A method of generating physically-based rendering (PBR) textures is disclosed. A 3D mesh input is received. One or more text prompts associated with the 3D mesh input are received. The 3D mesh input and the one or more text prompts are in put into a trained machine-learning model. Output data is received from the trained machine-learning model. The output data comprises material and texture information. The material and texture information enables rendered views of the 3D mesh with the output data to respond to changes in lighting in a real-time 3D rendering environment. ## Claims ### Claim 1 A system comprising: one or more computer processors; one or more computer memories; a set of instructions incorporated into the one or more computer memories, the set of instructions configuring the one or more computer processors to perform operations, the operations comprising: receiving a 3D mesh input; receiving one or more text prompts associated with the 3D mesh input; inputting the 3D mesh input and the one or more text prompts into a trained machine-learning model, wherein the trained machine-learning model includes a cross-domain attention mechanism that aligns the feature spaces of a plurality of parallel networks, the plurality of parallel networks including a red, green, blue (RGB) space trained on RGB images and a physically-based rendering (PBR) trained on PBR textures; and receiving output data from the trained machine-learning model, the output data comprising material and texture information, wherein the material and texture information enables rendered views of a 3D mesh with the output data to respond to changes in lighting in a real-time 3D rendering environment. ### Claim 2 (depends on 1) The system of claim 1, further comprising applying the output data to the 3D mesh input for use in a real-time 3D rendering environment. ### Claim 3 (depends on 1) The system of claim 1, wherein the trained machine-learning model comprises an artificial intelligence model trained using a dataset of annotated 3D objects. ### Claim 4 (depends on 3) The system of claim 3, wherein the dataset of annotated 3D objects includes a variety of view-independent material properties suitable for multi-view fusion in 3D software applications. ### Claim 5 (depends on 1) The system of claim 1, wherein the output data further comprises a set of material parameters including at least a base color, metallic property, and roughness property, which are used by a 3D rendering application to simulate realistic lighting interactions. ### Claim 6 (depends on 5) The system of claim 5, wherein the set of material parameters further includes a normal map that encodes small geometric details to increase realism during rendering, the normal map being defined in a geometry tangent space normal map to ensure consistency across texture atlas unwrapping. ### Claim 7 A non-transitory computer-readable storage medium storing a set of instructions that, when executed by one or more computer processors, causes the one or more computer processors to perform operations, the operations comprising: receiving a 3D mesh input; receiving one or more text prompts associated with the 3D mesh input; inputting the 3D mesh input and the one or more text prompts into a trained machine-learning model, wherein the trained machine-learning model includes a cross-domain attention mechanism that aligns the feature spaces of a plurality of parallel networks, the plurality of parallel networks including a red, green, blue (RGB) space trained on RGB images and a physically-based rendering (PBR) trained on PBR textures; and receiving output data from the trained machine-learning model, the output data comprising material and texture information, wherein the material and texture information enables rendered views of a 3D mesh with the output data to respond to changes in lighting in a real-time 3D rendering environment. ### Claim 8 (depends on 7) The non-transitory computer-readable storage medium of claim 7, further comprising applying the output data to the 3D mesh input for use in a real-time 3D rendering environment. ### Claim 9 (depends on 7) The non-transitory computer-readable storage medium of claim 7, wherein the trained machine-learning model comprises an artificial intelligence model trained using a dataset of annotated 3D objects. ### Claim 10 (depends on 9) The non-transitory computer-readable storage medium of claim 9, wherein the dataset of annotated 3D objects includes a variety of view-independent material properties suitable for multi-view fusion in 3D software applications. ### Claim 11 (depends on 7) The non-transitory computer-readable storage medium of claim 7, wherein the output data further comprises a set of material parameters including at least a base color, metallic property, and roughness property, which are used by a 3D rendering application to simulate realistic lighting interactions. ### Claim 12 (depends on 11) The non-transitory computer-readable storage medium of claim 11, wherein the set of material parameters further includes a normal map that encodes small geometric details to increase realism during rendering, the normal map being defined in a geometry tangent space normal map to ensure consistency across texture atlas unwrapping. ### Claim 13 A method comprising: receiving a 3D mesh input; receiving one or more text prompts associated with the 3D mesh input; inputting the 3D mesh input and the one or more text prompts into a trained machine-learning model, wherein the trained machine-learning model includes a cross-domain attention mechanism that aligns the feature spaces of a plurality of parallel networks, the plurality of parallel networks including a red, green, blue (RGB) space trained on RGB images and a physically-based rendering (PBR) trained on PBR textures; and receiving output data from the trained machine-learning model, the output data comprising material and texture information, wherein the material and texture information enables rendered views of a 3D mesh with the output data to respond to changes in lighting in a real-time 3D rendering environment. ### Claim 14 (depends on 13) The method of claim 13, further comprising applying the output data to the 3D mesh input for use in a real-time 3D rendering environment. ### Claim 15 (depends on 13) The method of claim 13, wherein the trained machine-learning model comprises an artificial intelligence model trained using a dataset of annotated 3D objects. ### Claim 16 (depends on 15) The method of claim 15, wherein the dataset of annotated 3D objects includes a variety of view-independent material properties suitable for multi-view fusion in 3D software applications. ### Claim 17 (depends on 13) The method of claim 13, wherein the output data further comprises a set of material parameters including at least a base color, metallic property, and roughness property, which are used by a 3D rendering application to simulate realistic lighting interactions.