Text-to-image diffusion models for generalizable mesh generation
AssigneeQUALCOMM Incorporated
InventorsAntoine Clement Mercier, Guillaume Jean Fernand Berger, Fatih Murat Porikli, Ramin Ebrahim Nakhli
Life of this patent
Estimating the term…
Abstract
Certain aspects of the present disclosure provide techniques and apparatus for improved machine learning. In an example method, a multiview latent tensor is generated based on processing a textual input using a diffusion machine learning model, where the multiview latent tensor corresponds to a plurality of orthographic projections corresponding to the textual input. A triplane latent tensor is generated based on the multiview latent tensor using a conversion machine learning model, and a three-dimensional mesh is generated based on processing the triplane latent tensor using a decoder machine learning model.
Claims20
A processing system comprising:
- one or more memories comprising processor-executable instructions;
- and
- one or more processors configured to execute the processor-executable instructions and cause the processing system to:
- generate a multiview latent tensor based on processing input using a diffusion machine learning model, wherein the multiview latent tensor corresponds to a plurality of orthographic projections corresponding to the input;
- generate a triplane latent tensor based on the multiview latent tensor using a conversion machine learning model, wherein to generate the triplane latent tensor, the one or more processors are configured to realign a first pair of orthographic projections in the multiview latent tensor based on aligning a first orthographic projection of the first pair of orthographic projections to match orientations with a second orthographic projection of the first pair of orthographic projections;
- and
- generate a three-dimensional mesh based on processing the triplane latent tensor using a decoder machine learning model.
A processor-implemented method of machine learning, comprising:
- generating a multiview latent tensor based on processing input using a diffusion machine learning model, wherein the multiview latent tensor corresponds to a plurality of orthographic projections corresponding to the input;
- generating a triplane latent tensor based on the multiview latent tensor using a conversion machine learning model, comprising realigning a first pair of orthographic projections in the multiview latent tensor based on aligning a first orthographic projection of the first pair of orthographic projections to match orientations with a second orthographic projection of the first pair of orthographic projections;
- and
- generating a three-dimensional mesh based on processing the triplane latent tensor using a decoder machine learning model.
One or more non-transitory computer-readable media comprising processor-executable instructions that, when executed by one or more processors of a processing system, cause the processing system to:
- generate a multiview latent tensor based on processing a textual input using a diffusion machine learning model, wherein the multiview latent tensor corresponds to a plurality of orthographic projections corresponding to the input;
- generate a triplane latent tensor based on the multiview latent tensor using a conversion machine learning model, comprising realigning a pair of orthographic projections in the multiview latent tensor based on aligning a first orthographic projection of the pair of orthographic projections to match orientations with a second orthographic projection of the pair of orthographic projections;
- and
- generate a three-dimensional mesh based on processing the triplane latent tensor using a decoder machine learning model.
Dependencies are inferred from the claim text.