# US 2025/0095221 A1 · Saliency-based background generation > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2025/0095221 A1 - Document ID: US20250095221A1 - Status: Published application - Kind: A1 - Application number: 18/369,924 - Filed: 2023-09-19 - Published: 2025-03-20 - Earliest claimed: 2023-09-19 - Assignees: Adobe Inc. - Applicants: Adobe Inc. - Inventors: Ajay Jain; Michele Saad; Irgelkha Mejia - Web page: https://patentgrep.com/patent/US20250095221A1 ## CPC - G06V 10/86: using syntactic or structural representations of the image or video pattern, e.g. symbolic string recognition; using graph matching - G06F 40/279: Recognition of textual entities - G06T 11/00: Two-dimensional [2D] image generation - G06T 11/60: Creating or editing images; Combining images with text - G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract In accordance with the described techniques, a background generation system receives one or more images depicting an object, and textual information describing the object. A generative text model is employed to generate a prompt based on the one or more images and the textual information. Further, a generative image model is employed to generate an output image. To do so, the generative image model generates a background image based on the prompt, and the object is incorporated into the background image. Using a visual saliency model, the background generation system determines a visual saliency defining a degree of fixation on the object within the output image. The background generation system outputs the output image based on the visual saliency meeting a threshold. ## Claims ### Claim 1 A method, comprising: receiving, by a processing device, one or more images depicting an object and textual information describing the object; generating, by the processing device and using a generative text model, a prompt based on the one or more images and the textual information; generating, by the processing device and using a generative image model, a background image based on the prompt; generating, by the processing device, an output image by incorporating the object into the background image; determining, by the processing device and using a visual saliency model, a visual saliency defining a degree of fixation on the object within the output image; and outputting, by the processing device, the output image based on the visual saliency meeting a threshold. ### Claim 2 (depends on 1) The method of claim 1, wherein the receiving the one or more images and the textual information includes obtaining the one or more images and the textual information from an online publication of the object. ### Claim 3 (depends on 2) The method of claim 2, wherein the textual information obtained from the online publication includes one or more of categories to which the object belongs, usages of the object, demographics the object is intended for, descriptions of the object, and reviews of the object. ### Claim 4 (depends on 2) The method of claim 2, the method further comprising obtaining, by the processing device and from a database of an online publication service hosting the online publication, user data describing one or more attributes of a user responsive to receiving a notification indicating that the user is requesting to access the online publication via the online publication service, the prompt generated based on the user data. ### Claim 5 (depends on 1) The method of claim 1, wherein the generating the prompt includes: querying a knowledge graph with the textual information, the knowledge graph indicating associations between entities; and extracting, using an entity extraction technique, one or more entities from the knowledge graph that correspond to the textual information, the prompt generated based on the one or more entities. ### Claim 6 (depends on 1) The method of claim 1, wherein the generating the prompt includes extracting image tags from the one or more images by processing the one or more images using a tag extraction model, the prompt generated based on the image tags. ### Claim 7 (depends on 1) The method of claim 1, wherein the generating the prompt includes generating, using a usage generation model, a usage of the object based on the one or more images and the textual information, the prompt generated based on the usage. ### Claim 8 (depends on 7) The method of claim 7, further comprising: receiving, by the processing device, training data including one or more training images depicting a training object, training textual information describing the training object, and a ground truth usage of the training object; generating, by the processing device and using the usage generation model, a predicted usage of the training object based on the one or more training images and the training textual information; and training, by the processing device, the usage generation model by comparing the predicted usage to the ground truth usage. ### Claim 9 (depends on 1) The method of claim 1, wherein the incorporating the object into the background image includes: generating an object image by separating the object from an image of the one or more images; and filling a background of the object image with the background image, the filled object image corresponding to the output image. ### Claim 10 (depends on 9) The method of claim 9, wherein the filling the background of the object image includes positioning content of the background image relative to the object. ### Claim 11 (depends on 1) The method of claim 1, wherein the determining the visual saliency includes: generating, using the visual saliency model, a saliency map of the output image; determining, using the saliency map, a first degree of fixation on a region of the output image that includes the object, and a second degree of fixation outside the region of the output image; and comparing the first degree of fixation to the second degree of fixation. ### Claim 12 (depends on 1) The method of claim 1, further comprising generating, by the processing device, a different output image by generating, using the generative image model, a different background image based on the prompt, and incorporating the object into the different background image, the different output image generated based on the visual saliency being below the threshold. ### Claim 13 A system, comprising: a processing device; and a computer-readable medium storing instructions that, responsive to execution by the processing device, cause the processing device to perform operations including: receiving multiple images depicting an object; extracting, using a tag extraction model, image tags from the multiple images; generating, using a generative text model, a prompt based on the image tags; generating, using a generative image model, a background image based on the prompt; generating an output image by incorporating the object into the background image; determining, using a visual saliency model, a visual saliency defining a degree of fixation on the object within the output image; and outputting the output image based on the visual saliency meeting a threshold. ### Claim 14 (depends on 13) The system of claim 13, wherein the receiving the multiple images includes obtaining the multiple images from an online publication of the object. ### Claim 15 (depends on 13) The system of claim 13, wherein the generating the prompt includes: querying a knowledge graph with the image tags, the knowledge graph indicating associations between entities; and extracting, using an entity extraction technique, one or more entities from the knowledge graph that correspond to the image tags, the prompt generated based on the one or more entities. ### Claim 16 (depends on 13) The system of claim 13, wherein the determining the visual saliency includes: generating, using the visual saliency model, a saliency map of the output image; determining, using the saliency map, a first degree of fixation on a region of the output image that includes the object, and a second degree of fixation outside the region of the output image; and comparing the first degree of fixation to the second degree of fixation. ### Claim 17 A non-transitory computer-readable medium storing executable instructions, which when executed by a processing device, cause the processing device to perform operations comprising: obtaining, from an online publication of an object, textual information describing the object; extracting, using an entity extraction technique, one or more entities that correspond to the textual information from a knowledge graph that indicates associations between entities; generating, using a generative text model, a prompt based on the one or more entities; generating, using a generative image model, a background image based on the prompt; generating an output image by incorporating the object into the background image; determining, using a visual saliency model, a visual saliency defining a degree of fixation on the object within the output image; and outputting the output image based on the visual saliency meeting a threshold. ### Claim 18 (depends on 17) The non-transitory computer-readable medium of claim 17, wherein the textual information obtained from the online publication includes one or more of categories to which the object belongs, usages of the object, demographics the object is intended for, descriptions of the object, and reviews of the object. ### Claim 19 (depends on 17) The non-transitory computer-readable medium of claim 17, the operations further including: receiving training data including training textual information describing training objects and corresponding ground truth usages of the training objects; training, using supervised learning, a usage generation model to predict usages for objects based on the training data; and generating, using the usage generation model, a usage based on the textual information, the prompt generated based on the usage. ### Claim 20 (depends on 17) The non-transitory computer-readable medium of claim 17, wherein the determining the visual saliency includes: generating, using the visual saliency model, a saliency map of the output image; determining, using the saliency map, a first degree of fixation on a region of the output image that includes the object, and a second degree of fixation outside the region of the output image; and comparing the first degree of fixation to the second degree of fixation.