# US 12,002,157 B2 · Variational auto-encoder for outputting a 3D model

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,002,157 B2
- Document ID: US12002157B2
- Status: Granted
- Kind: B2
- Application number: 17/408,258
- Filed: 2021-08-20
- Published: 2022-02-24
- Granted: 2024-06-04
- Earliest claimed: 2020-08-20
- Assignees: Dassault Systemes
- Applicants: Dassault Systemes
- Inventors: Nicolas Beltrand
- Web page: https://patentgrep.com/patent/US12002157B2

## CPC

- G06N 3/088: Non-supervised learning, e.g. competitive learning
- G06T 17/10: Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
- G06F 30/12: characterised by design entry means specially adapted for CAD, e.g. graphical user interfaces [GUI] specially adapted for CAD
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/047: Probabilistic or stochastic networks
- G06N 3/0475: Generative networks
- G06N 3/09: Supervised learning
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A computer-implemented method of machine-learning. The method includes obtaining a Variational Auto-encoder architecture for a neural network which is configured to take as an input a 2D sketch and a random variable, and to output a 3D model represented by the 2D sketch. The method of machine-learning also includes teaching the neural network.

## Claims

### Claim 1

A computer-implemented method of machine-learning comprising:
obtaining a Variational Auto-encoder architecture for a neural network which is configured to take as an input a 2D sketch and a random variable, and to output a 3D model represented by the 2D sketch; and
teaching the neural network,
wherein the 3D model is a parameterized 3D model defined by a set of parameters consisting of a first subset of one or more parameters and a second subset of one or more parameters, and the neural network is configured to selectively:
output a value for the set, and
take as input a value for the first subset from a user and output a value for the second subset, and
wherein the architecture includes a vector, the neural network being configured:
for selective outputting a value for the set, to produce a first value for the vector representing a probability distribution for the first subset, to output the value for the first subset corresponding to an argmax of the vector representing the probability distribution, and to further output the value for the second subset based on said first value for the vector; and
for selective taking as input a value for the first subset from the user and outputting the value of the second subset, to produce a second value for the vector having a value of 1 for coordinates corresponding to the value for the first subset and a value of 0 for all other coordinates, the neural network being configured to output the value for the second subset based on the second value for the vector.

### Claim 2 (depends on 1)

The computer-implemented method of machine-learning of claim 1, wherein the neural network includes:
a first part which is configured to take the 2D sketch and the random variable as inputs and to provide a respective output, and
a second part which is configured to take the respective output of the first part as input and to output the 3D model.

### Claim 3 (depends on 2)

The computer-implemented method of machine-learning of claim 2, wherein the first part includes:
a first subpart which is configured to take the 2D sketch as input and to output a first latent vector, and
a second subpart which is configured to take the first latent vector and the random variable, and to re-parameterize the first latent vector into a second latent vector with the random variable.

### Claim 4 (depends on 3)

The computer-implemented method of machine-learning of claim 3, wherein the second subpart is configured to perform the following operation:
output=μ+δ*ε
where:
output denotes the second latent vector,
ε denotes the random variable,
μ denotes a conditional mean of the second latent vector, and
δ denotes a conditional standard deviation of the second latent vector.

### Claim 5 (depends on 4)

The computer-implemented method of machine-learning of claim 4, wherein:
the random variable follows a probability distribution according to a law defined by a mean and a standard deviation, the random variable having a respective value for the mean and a respective value for the standard deviation, and
the teaching includes optimizing an objective function which comprises a loss, the loss penalizing a dissimilarity between:
the probability distribution according to the law as defined by μ and δ, and
the probability distribution according to the law as defined by said respective value of the random variable for the mean and said respective value of the random variable for the standard deviation.

### Claim 6 (depends on 1)

The computer-implemented method of machine-learning of claim 1, wherein the 3D model is defined by a section and an extrusion, the section being defined by a list of positional parameters, and the neural network comprises a recurrent neural network configured to output a value for the list.

### Claim 7 (depends on 6)

The computer-implemented method of machine-learning of claim 6, wherein the neural network is further configured to output a value for the extrusion based on a final state of the recurrent neural network.

### Claim 8 (depends on 7)

The computer-implemented method of machine-learning of claim 7, wherein the section is further defined by a number representing a type of the section, the neural network being further configured to output a vector representing a probability distribution for the number, and optionally the outputting of the value for the extrusion and/or for the list of positional parameters is further based on the vector representing the probability distribution.

### Claim 9 (depends on 1)

The computer-implemented method of machine-learning of claim 1, wherein the neural network includes:
a first part which is configured to take the 2D sketch and the random variable as inputs and to provide as output a respective latent vector, and
a second part which is configured to take the respective latent vector of the first part as input, the second part including:
a first subpart which is configured to output the vector representing the probability distribution, and
a second subpart which is configured to take as input a concatenation of the respective latent vector and the vector representing the probability distribution, and to output the second subset.

### Claim 10

A computer-implemented method of 3D design, the method of 3D design comprising:
obtaining a neural network taught according to a method of machine-learning including:
obtaining a Variational Auto-encoder architecture for a neural network which is configured to take as an input a 2D sketch and a random variable, and to output a 3D model represented by the 2D sketch, and
teaching the neural network;
obtaining a 2D sketch; and
applying the neural network to the 2D sketch and a random variable, to output a 3D model represented by the 2D sketch,
wherein the 3D model is a parameterized 3D model defined by a set of parameters consisting of a first subset of one or more parameters and a second subset of one or more parameters, and the neural network is configured to selectively:
output a value for the set, and
take as input a value for the first subset from a user and output a value for the second subset, and
wherein the architecture includes a vector, the neural network being configured:
for selective outputting a value for the set, to produce a first value for the vector representing a probability distribution for the first subset, to output the value for the first subset corresponding to an argmax of the vector representing the probability distribution, and to further output the value for the second subset based on said first value for the vector; and
for selective taking as input a value for the first subset from the user and outputting the value of the second subset, to produce a second value for the vector having a value of 1 for coordinates corresponding to the value for the first subset and a value of 0 for all other coordinates, the neural network being configured to output the value for the second subset based on the second value for the vector.

### Claim 11 (depends on 10)

The computer-implemented method of 3D design of claim 10, wherein the neural network includes:
a first part which is configured to take the 2D sketch and the random variable as inputs and to provide a respective output, and
a second part which is configured to take the respective output of the first part as input and to output the 3D model.

### Claim 12 (depends on 11)

The computer-implemented method of 3D design of claim 11, wherein the first part includes:
a first subpart which is configured to take the 2D sketch as input and to output a first latent vector, and
a second subpart which is configured to take the first latent vector and the random variable, and to re-parameterize the first latent vector into a second latent vector with the random variable.

### Claim 13 (depends on 12)

The computer-implemented method of 3D design of claim 12,
wherein the second subpart is configured to perform the following operation:
output=μ+δ*ε
where:
output denotes the second latent vector,
ε denotes the random variable,
μ denotes a conditional mean of the second latent vector, and
δ denotes a conditional standard deviation of the second latent vector.

### Claim 14 (depends on 13)

The computer-implemented method of 3D design of claim 13, wherein:
the random variable follows a probability distribution according to a law defined by a mean and a standard deviation, the random variable having a respective value for the mean and a respective value for the standard deviation, and
the teaching includes optimizing an objective function which comprises a loss, the loss penalizing a dissimilarity between:
the probability distribution according to the law as defined by μ and δ, and
the probability distribution according to the law as defined by said respective value of the random variable for the mean and said respective value of the random variable for the standard deviation.

### Claim 15

A device comprising:
a non-transitory memory having recorded thereon a computer program that when executed by a processor causes the processor to
be configured to:
obtain a Variational Auto-encoder architecture for a neural network which is configured to take as an input a 2D sketch and a random variable, and to output a 3D model represented by the 2D sketch, and teach the neural network, wherein the 3D model is a parameterized 3D model defined by a set of parameters consisting of a first subset of one or more parameters and a second subset of one or more parameters, and the neural network is configured to selectively:
output a value for the set, and take as input a value for the first subset from a user and output a value for the second subset, and
wherein the architecture includes a vector, the neural network being configured:
for selective outputting a value for the set, to produce a first value for the vector representing a probability distribution for the first subset, to output the value for the first subset corresponding to an argmax of the vector representing the probability distribution, and to further output the value for the second subset based on said first value for the vector; and
for selective taking as input a value for the first subset from the user and outputting the value of the second subset, to produce a second value for the vector having a value of 1 for coordinates corresponding to the value for the first subset and a value of 0 for all other coordinates, the neural network being configured to output the value for the second subset based on the second value for the vector, and/or
be configured to:
obtain a neural network taught by the processor being configured to:
obtain the Variational Auto-encoder architecture for the neural network which is configured to take as the input the 2D sketch and the random variable, and to output the 3D model represented by the 2D sketch, and
teach the neural network, provide a 2D sketch, and apply the neural network to the 2D sketch and a random variable, to output a 3D model represented by the 2D sketch, wherein the 3D model is a parameterized 3D model defined by a set of parameters consisting of a first subset of one or more parameters and a second subset of one or more parameters, and the neural network is configured to selectively:
output a value for the set, and take as input a value for the first subset from a user and output a value for the second subset, and wherein the architecture includes a vector, the neural network being configured:
for selective outputting a value for the set, to produce a first value for the vector representing a probability distribution for the first subset, to output the value for the first subset corresponding to an argmax of the vector representing the probability distribution, and to further output the value for the second subset based on said first value for the vector; and
for selective taking as input a value for the first subset from the user and outputting the value of the second subset, to produce a second value for the vector having a value of 1 for coordinates corresponding to the value for the first subset and a value of 0 for all other coordinates, the neural network being configured to output the value for the second subset based on the second value for the vector.

### Claim 16 (depends on 15)

The device of claim 15, wherein the neural network includes:
a first part which is configured to take the 2D sketch and the random variable as inputs and to provide a respective output, and
a second part which is configured to take the respective output of the first part as input and to output the 3D model.

### Claim 17 (depends on 16)

The device of claim 16, wherein the second part is configured to perform the following operation: output=μ+δ*ε,
where:
output denotes a second latent vector,
ε denotes the random variable,
μ denotes a conditional mean of the second latent vector, and
δ denotes a conditional standard deviation of the second latent vector.

### Claim 18 (depends on 17)

The device of claim 17, wherein:
the random variable follows a probability distribution according to a law defined by a mean and a standard deviation, the random variable having a respective value for the mean and a respective value for the standard deviation, and
the teaching comprises optimizing an objective function which comprises a loss, the loss penalizing a dissimilarity between:
the probability distribution according to the law as defined by μ and δ, and
the probability distribution according to the law as defined by said respective value of the random variable for the mean and said respective value of the random variable for the standard deviation.

### Claim 19 (depends on 15)

The device of claim 15, further comprising:
the processor, wherein
the processor is a central processing unit (CPU).

### Claim 20

A computer-implemented method of machine-learning comprising:
obtaining a Variational Auto-encoder architecture for a neural network which is configured to take as an input a 2D sketch and a random variable, and to output a 3D model represented by the 2D sketch; and
teaching the neural network,
wherein the neural network includes:
a first part which is configured to take the 2D sketch and the random variable as inputs and to provide a respective output, and
a second part which is configured to take the respective output of the first part as input and to output the 3D model,
wherein the first part includes:
a first subpart which is configured to take the 2D sketch as input and to output a first latent vector, and
a second subpart which is configured to take the first latent vector and the random variable, and to re-parameterize the first latent vector into a second latent vector with the random variable, and
wherein the second subpart is configured to perform the following operation:
output=μ+δ*ε
where:
output denotes the second latent vector,
ε denotes the random variable,
μ denotes a conditional mean of the second latent vector, and
δ denotes a conditional standard deviation of the second latent vector.

### Claim 21 (depends on 20)

The computer-implemented method of machine-learning of claim 20, wherein the 3D model is defined by a section and an extrusion, the section being defined by a list of positional parameters, and the neural network comprises a recurrent neural network configured to output a value for the list.

### Claim 22 (depends on 20)

A computer-implemented method of 3D design, the method of 3D design comprising:
obtaining a neural network taught according to the method of machine-learning according to claim 20.

### Claim 23

A computer-implemented method of machine-learning comprising:
obtaining a Variational Auto-encoder architecture for a neural network which is configured to take as an input a 2D sketch and a random variable, and to output a 3D model represented by the 2D sketch; and
teaching the neural network,
wherein the 3D model is defined by a section and an extrusion, the section being defined by a list of positional parameters, and the neural network comprises a recurrent neural network configured to output a value for the list,
wherein the neural network is further configured to output a value for the extrusion based on a final state of the recurrent neural network, and
wherein the section is further defined by a number representing a type of the section, the neural network being further configured to output a vector representing a probability distribution for the number, and optionally the outputting of the value for the extrusion and/or for the list of positional parameters is further based on the vector representing the probability distribution.

### Claim 24 (depends on 23)

A computer-implemented method of 3D design, the method of 3D design comprising:
obtaining a neural network taught according to the method of machine-learning according to claim 23.