# US 12,182,957 B2 · Neural style transfer in three-dimensional shapes > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,182,957 B2 - Document ID: US12182957B2 - Status: Granted - Kind: B2 - Application number: 18/149,601 - Filed: 2023-01-03 - Published: 2023-10-12 - Granted: 2024-12-31 - Earliest claimed: 2022-04-07 - Assignees: Autodesk, Inc. - Applicants: Autodesk, Inc. - Inventors: Hooman Shayani; Marco Fumero; Aditya Sanghi - Web page: https://patentgrep.com/patent/US12182957B2 ## CPC - G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts - G06N 3/0455: Auto-encoder networks; Encoder-decoder networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/047: Probabilistic or stochastic networks - G06N 3/0475: Generative networks - G06N 3/08: Learning methods - G06N 3/084: Backpropagation, e.g. using gradient descent - G06N 3/088: Non-supervised learning, e.g. competitive learning - G06N 3/092: Reinforcement learning - G06T 17/00: Three-dimensional [3D] modelling for computer graphics - G06T 17/10: Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes - G06T 2210/56: Particle system, point based geometry or rendering - G06T 2219/2021: Shape modification - G06T 2219/2024: Style variation - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract One embodiment of the present invention sets forth a technique for performing style transfer. The technique includes generating an input shape representation that includes a plurality of points near a surface of an input three-dimensional (3D) shape, where the input 3D shape includes content-based attributes associated with an object. The technique also includes determining a style code based on a difference between a first latent representation of a first 3D shape and a second latent representation of a second 3D shape, where the second 3D shape is generated by applying one or more augmentations to the first 3D shape. The technique further includes generating, based on the input shape representation and style code, an output 3D shape having the content-based attributes of the input 3D shape and style-based attributes associated with the style code, and generating a 3D model of the object based on the output 3D shape. ## Claims ### Claim 1 A computer-implemented method for performing style transfer, the method comprising: generating an input shape representation that includes a plurality of points in proximity to a surface of an input three-dimensional (3D) shape, wherein the input 3D shape includes one or more content-based attributes associated with an object; determining a style code based on a difference between a first latent representation of a first 3D shape and a second latent representation of a second 3D shape, wherein the second 3D shape is generated by applying one or more augmentations to the first 3D shape; executing a trained machine learning model based on the input shape representation and the style code to generate an output 3D shape having the one or more content-based attributes of the input 3D shape and one or more style-based attributes associated with the style code; and generating a 3D model of the object based on the output 3D shape. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, further comprising applying the one or more augmentations to a third 3D shape to generate the input 3D shape. ### Claim 3 (depends on 1) The computer-implemented method of claim 1, wherein executing the trained machine learning model comprises executing the trained machine learning model based on a latent vector corresponding to the style code. ### Claim 4 (depends on 1) The computer-implemented method of claim 1, wherein executing the trained machine learning model comprises executing a set of convolutional layers included in the trained machine learning model to generate multiple sets of features associated with multiple resolutions for each point included in the plurality of points. ### Claim 5 (depends on 1) The computer-implemented method of claim 1, wherein the output 3D shape is generated by generating a plurality of output values for the plurality of points based on a multi-scale feature representation associated with the plurality of points and the style code. ### Claim 6 (depends on 5) The computer-implemented method of claim 5, wherein the plurality of output values comprises a plurality of signed distance function values associated with the plurality of points. ### Claim 7 (depends on 1) The computer-implemented method of claim 1, wherein determining the style code comprises: converting the difference between the first latent representation and the second latent representation into a plurality of intermediate representations associated with the plurality of points; and performing one or more pooling operations on the plurality of intermediate representations. ### Claim 8 (depends on 1) The computer-implemented method of claim 1, wherein the input shape representation further comprises a plurality of signed distance function values associated with a grid of points disposed around the input 3D shape. ### Claim 9 (depends on 1) The computer-implemented method of claim 1, wherein the one or more augmentations comprise at least one of a smoothing augmentation or a coarsening augmentation. ### Claim 10 (depends on 1) The computer-implemented method of claim 1, wherein the first 3D shape comprises at least one of a textual character, a piece of furniture, a car, or an object generated via a generative design process. ### Claim 11 One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of: generating an input shape representation that includes a plurality of points in proximity to a surface of an input three-dimensional (3D) shape, wherein the input 3D shape includes one or more content-based attributes associated with an object; determining a style code based on a first latent representation of a first 3D shape and a second latent representation of a second 3D shape, wherein the second 3D shape is generated by applying one or more augmentations to the first 3D shape; executing a trained machine learning model based on the input shape representation and the style code to generate an output 3D shape having the one or more content-based attributes of the input 3D shape and one or more style-based attributes associated with the style code; and generating a 3D model of the object based on the output 3D shape. ### Claim 12 (depends on 11) The one or more non-transitory computer-readable media of claim 11, wherein the instructions further cause the one or more processors to perform the step of applying the one or more augmentations to a third 3D shape to generate the input 3D shape. ### Claim 13 (depends on 11) The one or more non-transitory computer-readable media of claim 11, wherein determining the style code comprises: computing a difference between the first latent representation and the second latent representation; and applying a set of neural network layers to the difference to generate the style code. ### Claim 14 (depends on 11) The one or more non-transitory computer-readable media of claim 11, wherein executing the trained machine learning model comprises: executing an encoder neural network included in the trained machine learning model to convert the input shape representation into a shape signature associated with the input 3D shape, wherein the shape signature includes multiple sets of features associated with multiple resolutions for each point included in the plurality of points; and executing a decoder neural network included in the trained machine learning model to convert the shape signature and the style code into the output 3D shape. ### Claim 15 (depends on 14) The one or more non-transitory computer-readable media of claim 14, wherein executing the decoder neural network comprises: inputting a concatenation of the shape signature and the style code into the decoder neural network; and generating, via execution of the decoder neural network, a plurality of output values for the plurality of points based on the concatenation of the shape signature and the style code. ### Claim 16 (depends on 14) The one or more non-transitory computer-readable media of claim 14, wherein executing the encoder neural network comprises: generating multiple grids of features associated with the multiple resolutions based on a set of values determined for a grid of points disposed around the input 3D shape; and for each grid of features included in the multiple grids of features and each point included in the plurality of points, performing one or more interpolation operations on the grid of features to generate a corresponding set of features for the point. ### Claim 17 (depends on 11) The one or more non-transitory computer-readable media of claim 11, wherein the style code is determined based on a difference between the first latent representation and the second latent representation. ### Claim 18 (depends on 11) The one or more non-transitory computer-readable media of claim 11, wherein the one or more augmentations comprise at least one of a smoothing augmentation or a coarsening augmentation. ### Claim 19 (depends on 11) The one or more non-transitory computer readable media of claim 11, wherein the input shape representation further comprises a set of occupancy values associated with a grid of points disposed around the input 3D shape. ### Claim 20 A system, comprising: one or more memories that store instructions, and one or more processors that are coupled to the one or more memories and, when executing the instructions, are configured to perform the steps of: generating an input shape representation that includes a plurality of points in proximity to a surface of an input three-dimensional (3D) shape, wherein the input 3D shape includes one or more content-based attributes associated with an object; determining a style code corresponding to a difference between a first latent representation of a first 3D shape and a second latent representation of a second 3D shape, wherein the second 3D shape is generated by applying one or more augmentations to the first 3D shape; executing a machine learning model based on the input shape representation and the style code to generate an output 3D shape having the one or more content-based attributes of the input 3D shape and one or more style-based attributes associated with the style code; and generating a 3D model of the object based on the output 3D shape.