Interactive three-dimension aware text-to-image generation
AssigneeAdobe Inc.
InventorsMatheus Gadelha, Tomasz Opasinski, Kevin Blackburn-Matzen, Mathieu Kevin Pascal Gaillard, Giorgio Gori, Radomir Mech
Life of this patent
Estimating the term…
Abstract
An image processing system is configured to receive a three-dimensional (3D) model and a text prompt that describes a scene corresponding to the 3D model. The system may then generate a depth map of the 3D model and generate an output image based on the depth map and the text prompt. The output image may depicts a view of the scene that includes textures described by the text prompt. The output image may be generated using an image generation model.
Claims20
A method comprising:
- displaying, via a three-dimensional (3D) modeling user interface (UI) element of a combined interface, a 3D model of an object;
- receiving, via a 3D editing UI element of the combined interface, a 3D edit input indicating a change to the 3D model;
- displaying, via the 3D modeling UI element of the combined interface, a modified 3D model depicting a modified object including the change indicated by the 3D edit input;
- receiving, via a text UI element of the combined interface, a text prompt describing a scene including the object and a texture for the object;
- and
- displaying, via a preview element of the combined interface, an output image in response to receiving the text prompt, wherein the output depicts the scene including the modified object with the texture from the text prompt.
An apparatus comprising:
- at least one processor;
- at least one memory storing instructions and in electronic communication with the at least one processor;
- a three-dimensional (3D) modeling user interface (UI) element of a combined interface configured to display a 3D model of an object;
- a 3D editing UI element of the combined interface configured to receive a 3D edit input indicating a change to the 3D model and display a modified 3D model depicting a modified object including the change indicated by the 3D edit input;
- a text UI element of the combined interface configured to receive a text prompt describing a scene including the object and a texture for the object;
- and
- a preview element of the combined interface configured to display an output image in response to receiving the text prompt, wherein the output depicts the scene including the modified object with the texture from the text prompt.
A non-transitory computer readable medium storing code for image processing, the code comprising instructions executable by a processor to:
- display, via a three-dimensional (3D) modeling user interface (UI) element of a combined interface, a 3D model of an object;
- receive, via a 3D editing UI element of the combined interface, a 3D edit input indicating a change to the 3D model;
- display, via the 3D modeling UI element of the combined interface, a modified 3D model depicting a modified object including the change indicated by the 3D edit input;
- receive, via a text UI element of the combined interface, a text prompt describing a scene including the object and a texture for the object;
- and
- display, via a preview element of the combined interface, an output image in response to receiving the text prompt, wherein the output depicts the scene including the modified object with the texture from the text prompt.
Dependencies are inferred from the claim text.