# US 12,182,958 B2 · Training machine learning models to perform neural style transfer in three-dimensional shapes > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,182,958 B2 - Document ID: US12182958B2 - Status: Granted - Kind: B2 - Application number: 18/149,605 - Filed: 2023-01-03 - Published: 2023-10-12 - Granted: 2024-12-31 - Earliest claimed: 2022-04-07 - Assignees: Autodesk, Inc. - Applicants: Autodesk, Inc. - Inventors: Hooman Shayani; Marco Fumero; Aditya Sanghi - Web page: https://patentgrep.com/patent/US12182958B2 ## CPC - G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts - G06N 3/0455: Auto-encoder networks; Encoder-decoder networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/047: Probabilistic or stochastic networks - G06N 3/0475: Generative networks - G06N 3/08: Learning methods - G06N 3/084: Backpropagation, e.g. using gradient descent - G06N 3/088: Non-supervised learning, e.g. competitive learning - G06N 3/092: Reinforcement learning - G06T 17/00: Three-dimensional [3D] modelling for computer graphics - G06T 17/10: Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes - G06T 2210/56: Particle system, point based geometry or rendering - G06T 2219/2021: Shape modification - G06T 2219/2024: Style variation - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract One embodiment of the present invention sets forth a technique for training a machine learning model to perform style transfer. The technique includes applying one or more augmentations to a first input three-dimensional (3D) shape to generate a second input 3D shape. The technique also includes generating, via a first set of neural network layers, a style code based on a first latent representation of the first input 3D shape and a second latent representation of the second input 3D shape. The technique further includes generating, via a second set of neural network layers, a first output 3D shape based on the style code and the second latent representation, and performing one or more operations on the first and second sets of neural network layers based on a first loss associated with the first output 3D shape to generate a trained machine learning model. ## Claims ### Claim 1 A computer-implemented method for training a machine learning model to perform style transfer, the method comprising: applying one or more augmentations to a first input three-dimensional (3D) shape to generate a second input 3D shape; generating, via a first set of neural network layers, a style code based on a first latent representation of the first input 3D shape and a second latent representation of the second input 3D shape; generating, via a second set of neural network layers, a first output 3D shape based on the style code and the second latent representation; and performing one or more operations on the first set of neural network layers and the second set of neural network layers based on a first loss associated with the first output 3D shape to generate a trained machine learning model. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, further comprising generating, via the second set of neural network layers, a second output 3D shape based on the first latent representation, wherein the trained machine learning model is further generated based on a second loss associated with the second output 3D shape. ### Claim 3 (depends on 2) The computer-implemented method of claim 2, further comprising computing the second loss between the second output 3D shape and the first input 3D shape. ### Claim 4 (depends on 1) The computer-implemented method of claim 1, further comprising generating, via the second set of neural network layers, a second output 3D shape based on the second latent representation, wherein the trained machine learning model is further generated based on a second loss associated with the second output 3D shape. ### Claim 5 (depends on 4) The computer-implemented method of claim 4, further comprising computing the second loss between the second output 3D shape and the second input 3D shape. ### Claim 6 (depends on 1) The computer-implemented method of claim 1, further comprising computing the first loss between the first output 3D shape and the first input 3D shape. ### Claim 7 (depends on 1) The computer-implemented method of claim 1, wherein generating the style code comprises: generating, via an encoder neural network, the first latent representation corresponding to a first multi-scale feature representation of a first plurality of points in proximity to a surface of the first input 3D shape; and generating, via the encoder neural network, the second latent representation corresponding to a second multi-scale feature representation of a second plurality of points in proximity to a surface of the second input 3D shape. ### Claim 8 (depends on 1) The computer-implemented method of claim 1, wherein generating the style code comprises: inputting an aggregation of the first latent representation and the second latent representation into the first set of neural network layers; and executing the first set of neural network layers to generate a latent vector corresponding to the style code. ### Claim 9 (depends on 1) The computer-implemented method of claim 1, wherein generating the first output 3D shape comprises: inputting the style code and the second latent representation into the second set of neural network layers; and executing the second set of neural network layers to generate a set of signed distance function values corresponding to the first output 3D shape. ### Claim 10 (depends on 1) The computer-implemented method of claim 1, wherein the first loss comprises a reconstruction loss. ### Claim 11 One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of: applying one or more augmentations to a first input three-dimensional (3D) shape to generate a second input 3D shape; generating, via a first set of neural network layers, a style code based on a first latent representation of the first input 3D shape and a second latent representation of the second input 3D shape; generating, via a second set of neural network layers, a first output 3D shape based on the style code and the second latent representation; and performing one or more operations on the first set of neural network layers and the second set of neural network layers based on a first loss associated with the first output 3D shape to generate a trained machine learning model. ### Claim 12 (depends on 11) The one or more non-transitory computer-readable media of claim 11, wherein the instructions further cause the one or more processors to perform the steps of: generating, via the second set of neural network layers, a second output 3D shape based on the first latent representation, wherein the trained machine learning model is further generated based on a second loss associated with the second output 3D shape. ### Claim 13 (depends on 12) The one or more non-transitory computer-readable media of claim 12, wherein the instructions further cause the one or more processors to perform the step of computing the second loss as a first reconstruction loss between the second output 3D shape and the first input 3D shape. ### Claim 14 (depends on 12) The one or more non-transitory computer-readable media of claim 12, wherein the instructions further cause the one or more processors to perform the steps of generating, via the second set of neural network layers, a third output 3D shape based on the second latent representation, wherein the trained machine learning model is further generated based on a second reconstruction loss associated with the third output 3D shape. ### Claim 15 (depends on 14) The one or more non-transitory computer-readable media of claim 14, wherein the instructions further cause the one or more processors to perform the step of computing the third loss as a second reconstruction loss between the third output 3D shape and the second input 3D shape. ### Claim 16 (depends on 11) The one or more non-transitory computer-readable media of claim 11, wherein the instructions further cause the one or more processors to perform the step of computing the first loss as a reconstruction loss between the first output 3D shape and the first input 3D shape. ### Claim 17 (depends on 11) The one or more non-transitory computer-readable media of claim 11, wherein generating the style code comprises: generating, via a third set of neural network layers, the first latent representation corresponding to a first multi-scale feature representation of a first plurality of points in proximity to a surface of the first input 3D shape; generating, via the third set of neural network layers, the second latent representation corresponding to a second multi-scale feature representation of a second plurality of points in proximity to a surface of the second input 3D shape; and inputting the first latent representation and the second latent representation into the first set of neural network layers. ### Claim 18 (depends on 17) The one or more non-transitory computer-readable media of claim 17, wherein generating the first latent representation comprises: inputting a first grid of values associated with the first input 3D shape into the third set of neural network layers; generating, via execution of the third set of neural network layers, multiple grids of features from the first grid of values; and for each grid of features included in the multiple grids of features, interpolating a subset of feature values included in the grid of features into a set of feature values for a point sampled in proximity to a surface of the first input 3D shape. ### Claim 19 (depends on 11) The one or more non-transitory computer-readable media of claim 11, wherein generating the first output 3D shape comprises: inputting a concatenation of the style code and the second latent representation into the second set of neural network layers; and executing the second set of neural network layers to generate a set of signed distance function values corresponding to the first output 3D shape. ### Claim 20 A system, comprising: one or more memories that store instructions, and one or more processors that are coupled to the one or more memories and, when executing the instructions, are configured to perform the steps of: applying one or more augmentations to a first input three-dimensional (3D) shape to generate a second input 3D shape; generating, via a first set of neural network layers, a style code based on a first latent representation of the first input 3D shape and a second latent representation of the second input 3D shape; generating, via a second set of neural network layers, a first output 3D shape based on the style code and the second latent representation; and performing one or more operations on the first set of neural network layers and the second set of neural network layers based on a first loss associated with the first output 3D shape to generate a trained machine learning model.