Image generation with legible scene text
AssigneeAdobe Inc.
InventorsNipun Jindal, Brent Getlin, Oliver Brdiczka
Life of this patent
Estimating the term…
Abstract
Systems and methods for generating images with legible scene text are described. Embodiments are configured to obtain a prompt describing a scene, where the prompt includes scene text indicating text that is intended to be shown in a generated image; encode, using a prompt encoder, the prompt to generate a prompt embedding; encode, using a character-level encoder, the scene text to generate a character-level embedding; and generate, using an image generation network, an image that includes the scene text based on the prompt embedding and the character-level embedding.
Claims20
A method comprising:
- obtaining a prompt describing a scene, wherein the prompt includes scene text;
- encoding, using a prompt encoder, the prompt to generate a prompt embedding;
- encoding, using a character-level encoder, the scene text to generate a character-level embedding;
- and
- generating, using an image generation network, an image that includes the scene text based on the prompt embedding and the character-level embedding.
A method comprising:
- obtaining training data including a prompt describing a scene, wherein the prompt includes scene text;
- and
- training, using the training data, an image generation network to generate an image that includes the scene text based on a prompt embedding of the prompt and a character-level embedding of the scene text.
An apparatus comprising:
- at least one processor;
- at least one memory including instructions executable by the processor;
- and
- the apparatus further comprising an image generation network comprising parameters stored in the at least one memory, wherein the image generation network is trained to generate an image that includes scene text based on a prompt embedding of a prompt that includes the scene text and based on a character-level embedding of the scene text.
Dependencies are inferred from the claim text.