Interactive three-dimension aware text-to-image generation
AssigneeAdobe Inc.
InventorsMatheus Gadelha, Tomasz Opasinski, Kevin James Blackburn-Matzen, Mathieu Kevin Pascal Gaillard, Giorgio Gori, Radomir Mech
Life of this patent
Estimating the term…
Abstract
An image processing system is configured to receive a three-dimensional (3D) model and a text prompt that describes a scene corresponding to the 3D model. The system may then generate a depth map of the 3D model and generate an output image based on the depth map and the text prompt. The output image may depicts a view of the scene that includes textures described by the text prompt. The output image may be generated using an image generation model.
Claims20
A method comprising:
- receiving a three-dimensional (3D) model and a text prompt that describes a scene corresponding to the 3D model;
- generating a depth map of the 3D model;
- and
- generating, by an image generation model, an output image based on the depth map and the text prompt, wherein the output image depicts a view of the scene.
An apparatus comprising:
- at least one processor;
- at least one memory storing instructions and in electronic communication with the at least one processor;
- a three-dimensional (3D) modeling application configured to generate a depth map of a 3D model;
- an image generation model configured to generate an output image based on the depth map and a text prompt, wherein the output image depicts a view of a scene described by the text prompt.
A non-transitory computer readable medium storing code for image processing, the code comprising instructions executable by a processor to:
- receive a 3D model and a text prompt that describes a scene corresponding to the 3D model;
- generate a depth map of the 3D model;
- and
- generate, using an image generation model, an output image based on the depth map and the text prompt, wherein the output image depicts a view of the scene.
Dependencies are inferred from the claim text.