Canonicalized codebook for 3D object generation

A computer-implemented method of applying a decoder and a codebook learnable according to a machine-learning including obtaining a training dataset of 3D models of real-world objects and learning, based on the training dataset and on a patch-decomposition of the 3D models of the training dataset, a finite codebook of quantized vectors and a neural network, the neural network including:

• a rotation-invariant encoder configured for rotation-invariant encoding of a patch of a 3D model into a quantized latent vector of the codebook, and a decoder configured for decoding a sequence of quantized latent vectors of the codebook, the sequence corresponding to a patch-decomposition, into a 3D model, the method comprising:
• obtaining a sequence of quantized latent vectors of the codebook;
• and
• applying the decoder to the sequence.

A device comprising:

• a non-transitory computer-readable data storage medium having recorded thereon a computer program having instructions for
• performing machine-learning that when executed by a processor causes the processor to be configured to:
• obtain a training dataset of 3D models of real-world objects, and
• learn, based on the training dataset and on a patch-decomposition of the 3D models of the training dataset, a finite codebook of quantized vectors and a neural network, the neural network including:
• a rotation-invariant encoder configured for rotation-invariant encoding of a patch of a 3D model into a quantized latent vector of the codebook, and a decoder configured for decoding a sequence of quantized latent vectors of the codebook, the sequence corresponding to a patch-decomposition, into a 3D model, and/or
• applying a decoder and a codebook learnable according to the machine-learning that when executed by the processor causes the processor to be configured to:
• obtain a sequence of quantized latent vectors of the codebook, and
• apply the decoder to the sequence.