# US 12,602,847 B2 · Systems and methods for layered image generation > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,602,847 B2 - Document ID: US12602847B2 - Status: Granted - Kind: B2 - Application number: 18/240,836 - Filed: 2023-08-31 - Published: 2025-03-06 - Granted: 2026-04-14 - Earliest claimed: 2023-08-31 - Assignees: Adeia Imaging LLC - Applicants: Adeia Imaging LLC - Inventors: Jean-Yves Couleaud; Ning Xu; Zhiyun Li - Web page: https://patentgrep.com/patent/US12602847B2 ## CPC - G06T 11/10: Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77) - G06T 7/11: Region-based segmentation - G06T 7/13: Edge detection - G06T 7/194: involving foreground-background segmentation - G06T 11/60: Creating or editing images; Combining images with text - G06T 2200/24: involving graphical user interfaces [GUIs] - G06T 2207/20081: Training; Learning - G06T 2207/20221: Image fusion; Image merging - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Systems and methods are described for generating, using the first trained machine learning model and based on text input, a single-layer image comprising a plurality of objects; generating a plurality of masks associated with the plurality of objects; determining a plurality of attributes associated with the plurality of objects; generating, using a second trained machine learning model, a plurality of textual descriptions respectively corresponding to the plurality of objects; inputting the plurality of textual descriptions, and the plurality of attributes, to the first trained machine learning model; generating, using the first trained machine learning model, a plurality of images respectively corresponding to the plurality of textual descriptions; and generating the multi-layer image by combining the plurality of images and by using the plurality of masks, wherein the plurality of images respectively correspond to a plurality of layers of the multi-layer image. ## Claims ### Claim 1 A computer-implemented method for generating a multi-layer image based on text input, the method comprising: generating, using a first trained machine learning model and based on the text input, a single-layer image comprising a plurality of objects; generating a plurality of masks associated with the plurality of objects; determining a plurality of attributes associated with the plurality of objects; generating, using a second trained machine learning model, a plurality of textual descriptions respectively corresponding to the plurality of objects; inputting the plurality of textual descriptions generated by the second trained machine learning model to the first trained machine learning model; generating, using the first trained machine learning model and based on the plurality of textual descriptions generated by the second trained machine learning model and the plurality of attributes, a plurality of images respectively corresponding to the plurality of textual descriptions; and generating the multi-layer image by combining the plurality of images and by using the plurality of masks, wherein the plurality of images respectively correspond to a plurality of layers of the multi-layer image. ### Claim 2 (depends on 1) The method of claim 1, wherein the second trained machine learning model comprises an image-to-text machine learning model, the plurality of images comprises a second plurality of images, and the method further comprises: segmenting the single-layer image to generate a first plurality of images, each image of the first plurality of images comprising a depiction of a respective object of the plurality of objects of the single-layer image; and inputting the first plurality of images to the second trained machine learning model, to generate the plurality of textual descriptions respectively corresponding to the plurality of objects. ### Claim 3 (depends on 2) The method of claim 2, further comprising: determining that, as a result of the segmenting, each respective image of the first plurality of images comprises at least one empty region at a portion of the respective image at which one of more objects of the plurality of objects is depicted in the single-layer image; and generating a mask for each respective empty region of the empty regions of the plurality of images to obtain the plurality of masks associated with the plurality of objects; and using the plurality of masks to segment, and add transparency to one or more portions of, the second plurality of images, wherein the multi-layer image is generated based at least in part on the segmented second plurality of images. ### Claim 4 (depends on 2) The method of claim 2, wherein: determining the plurality of attributes associated with the plurality of objects comprises detecting, for each image of the first plurality of images, edges of one or more objects in the image; inputting the plurality of textual descriptions to the first trained machine learning model comprises, for each image of the first plurality of images, inputting to the first trained machine learning model information indicative of the detected edges of the one or more objects in the image along with the corresponding textual description of the plurality of textual descriptions, wherein the generating of the second plurality of images is guided by the information indicative of the detected edges. ### Claim 5 (depends on 2) The method of claim 2, further comprising: determining that, as a result of the segmenting, each respective image of the first plurality of images comprises at least one empty region at a portion of the respective image at which one of more objects of the plurality of objects is depicted in the single-layer image; and prior to inputting the first plurality of images to the second trained machine learning model, modifying a size or a shape of one or more empty regions of the at least one empty region. ### Claim 6 (depends on 2) The method of claim 2, further comprising: determining that, as a result of the segmenting, each respective image of the first plurality of images comprises at least one empty region at a portion of the respective image at which one of more objects of the plurality of objects is depicted in the single-layer image; determining whether a size of each empty region exceeds a threshold; and in response to determining that the size of each empty region exceeds the threshold, performing: the generating of the plurality of textual descriptions respectively corresponding to the plurality of objects; the inputting of the plurality of textual descriptions to the first trained machine learning model; and the generating of the second plurality of images respectively corresponding to the plurality of textual descriptions. ### Claim 7 (depends on 2) The method of claim 2, wherein the plurality of textual descriptions comprise a second plurality of textual descriptions, and generating, using the second trained machine learning model, the second plurality of textual descriptions respectively corresponding to the plurality of objects further comprises: generating, using the second trained machine learning model, a first plurality of textual descriptions, based on the second trained machine learning model receiving input of the first plurality of images; and modifying the first plurality of textual descriptions based on the text input to generate the second plurality of textual descriptions. ### Claim 8 (depends on 2) The method of claim 2, wherein the second trained machine learning model comprises a large language model, and generating, using the second trained machine learning model, the plurality of textual descriptions respectively corresponding to the plurality of objects comprises: generating, using the second trained machine learning model and based on the text input, the plurality of textual descriptions. ### Claim 9 (depends on 1) The method of claim 1, further comprising: generating a depth map for the single-layer image, wherein the generating the multi-layer image further comprises ordering the plurality of images, respectively corresponding to the plurality of layers of the multi-layer image, based on the depth map. ### Claim 10 (depends on 1) The method of claim 1, further comprising: receiving input of a particular image, wherein the particular image is included as an object of the plurality of objects in the generated single-layer image based on the received input of the particular image; generating, for display at a graphical user interface, the multi-layer image, wherein the graphical user interface comprises one or more options to modify the multi-layer image; receiving selection of the one or more options; and modifying the multi-layer image based on the received selection. ### Claim 11 (depends on 1) The method of claim 1, further comprising: generating a plurality of variations of the multi-layer image based on the plurality of images. ### Claim 12 (depends on 11) The method of claim 11, wherein the plurality of variations comprises a first variation and a second variation, and one or more of a size, location, or appearance of a first object of the plurality of objects in the first variation is different from one or more of a size, location, or appearance of the first object in the second variation. ### Claim 13 A computer-implemented system for generating a multi-layer image based on text input, the system comprising: input/output (I/O) circuitry; control circuitry configured to: generate, using a first trained machine learning model and based on the text input received via the I/O circuitry, a single-layer image comprising a plurality of objects; generate a plurality of masks associated with the plurality of objects; determine a plurality of attributes associated with the plurality of objects; generate, using a second trained machine learning model, a plurality of textual descriptions respectively corresponding to the plurality of objects; input the plurality of textual descriptions generated by the second trained machine learning model to the first trained machine learning model; generate, using the first trained machine learning model and based on the plurality of textual descriptions generated by the second trained machine learning model and the plurality of attributes, a plurality of images respectively corresponding to the plurality of textual descriptions; and generate the multi-layer image by combining the plurality of images and by using the plurality of masks, wherein the plurality of images respectively correspond to a plurality of layers of the multi-layer image. ### Claim 14 (depends on 13) The system of claim 13, wherein the second trained machine learning model comprises an image-to-text machine learning model, the plurality of images comprises a second plurality of images, and the control circuitry is further configured to: segment the single-layer image to generate a first plurality of images, each image of the first plurality of images comprising a depiction of a respective object of the plurality of objects of the single-layer image; and input the first plurality of images to the second trained machine learning model, to generate the plurality of textual descriptions respectively corresponding to the plurality of objects. ### Claim 15 (depends on 14) The system of claim 14, wherein the control circuitry is further configured to: determine that, as a result of the segmenting, each respective image of the first plurality of images comprises at least one empty region at a portion of the respective image at which one of more objects of the plurality of objects is depicted in the single-layer image; and generate a mask for each respective empty region of the empty regions of the plurality of images to obtain the plurality of masks associated with the plurality of objects; and use the plurality of masks to segment, and add transparency to one or more portions of, the second plurality of images, wherein the multi-layer image is generated based at least in part on the segmented second plurality of images. ### Claim 16 (depends on 14) The system of claim 14, wherein the control circuitry is configured to: determine the plurality of attributes associated with the plurality of objects by detecting, for each image of the first plurality of images, edges of one or more objects in the image; input the plurality of textual descriptions to the first trained machine learning model by, for each image of the first plurality of images, inputting to the first trained machine learning model information indicative of the detected edges of the one or more objects in the image along with the corresponding textual description of the plurality of textual descriptions, wherein the generating of the second plurality of images is guided by the information indicative of the detected edges. ### Claim 17 (depends on 14) The system of claim 14, wherein the control circuitry is configured to: determine that, as a result of the segmenting, each respective image of the first plurality of images comprises at least one empty region at a portion of the respective image at which one of more objects of the plurality of objects is depicted in the single-layer image; and prior to inputting the first plurality of images to the second trained machine learning model, modify a size or a shape of one or more empty regions of the at least one empty region. ### Claim 18 (depends on 14) The system of claim 14, wherein the control circuitry is configured to: determine that, as a result of the segmenting, each respective image of the first plurality of images comprises at least one empty region at a portion of the respective image at which one of more objects of the plurality of objects is depicted in the single-layer image; determine whether a size of each empty region exceeds a threshold; and in response to determining that the size of each empty region exceeds the threshold, perform: the generating of the plurality of textual descriptions respectively corresponding to the plurality of objects; the inputting of the plurality of textual descriptions to the first trained machine learning model; and the generating of the second plurality of images respectively corresponding to the plurality of textual descriptions. ### Claim 19 (depends on 14) The system of claim 14, wherein the plurality of textual descriptions comprise a second plurality of textual descriptions, and the control circuitry is configured to generate, using the second trained machine learning model, the second plurality of textual descriptions respectively corresponding to the plurality of objects by: generating, using the second trained machine learning model, a first plurality of textual descriptions, based on the second trained machine learning model receiving input of the first plurality of images; and modifying the first plurality of textual descriptions based on the text input to generate the second plurality of textual descriptions. ### Claim 20 (depends on 14) The system of claim 14, wherein the second trained machine learning model comprises a large language model, and the control circuitry is configured to generate, using the second trained machine learning model, the plurality of textual descriptions respectively corresponding to the plurality of objects by: generating, using the second trained machine learning model and based on the text input, the plurality of textual descriptions.