# US 2026/0065442 A1 · Inpainting prompt generation using object prediction

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0065442 A1
- Document ID: US20260065442A1
- Status: Published application
- Kind: A1
- Application number: 18/819,609
- Filed: 2024-08-29
- Published: 2026-03-05
- Earliest claimed: 2024-08-29
- Assignees: Adobe Inc.
- Applicants: Adobe Inc.
- Inventors: Mang Tik Chiu; Yuqian Zhou; Lingzhi Zhang; Zhe Lin; Connelly Stuart Barnes; Sohrab Amirghodsi; Elya Shechtman
- Web page: https://patentgrep.com/patent/US20260065442A1

## CPC

- G06T 5/60: using machine learning, e.g. neural networks
- G06T 5/77: Retouching; Inpainting; Scratch removal
- G06F 40/30: Semantic analysis
- G06N 3/045: Combinations of networks
- G06N 3/0475: Generative networks
- G06T 7/10: Segmentation; Edge detection (motion-based segmentation G06T7/215)
- G06V 10/30: Noise filtering
- G06V 10/40: Extraction of image or video features
- G06V 10/70: using pattern recognition or machine learning (optical pattern recognition or electronic computations therefor G06V10/88)
- G06V 10/80: Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level (multimodal speaker identification or verification G10L17/10)
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method, apparatus, non-transitory computer readable medium, and system for generating suggested inpainting prompts include first obtaining an image depicting a first element. Embodiments then generate, using an embedding generation model, a text embedding based on the image and a noise input, where the text embedding represents the first element from the first image and a second element generated by the embedding generation model based on the noise input. Subsequently, embodiments generate a text prompt that includes the first element and the second element based on the text embedding.

## Claims

### Claim 1

A method comprising:
obtaining an image depicting a first element;
generating, using an embedding generation model, a text embedding based on the image and a noise input, wherein the text embedding represents the first element from the first image and a second element generated based on the noise input; and
generating a text prompt based on the text embedding, wherein the text prompt includes the first element and the second element.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
obtaining a mask indicating a region of the image, wherein the second element is based on the mask.

### Claim 3 (depends on 2)

The method of claim 2, further comprising:
generating a local image that includes the region indicated by the mask and excludes a background region of the image, wherein the text embedding is generated based on the image and the local image.

### Claim 4 (depends on 1)

The method of claim 1, wherein:
the second element is not depicted in the image.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
encoding the image to obtain an image embedding, wherein the text embedding is generated based on the image embedding.

### Claim 6 (depends on 5)

The method of claim 5, wherein:
the image embedding represents the first element and not the second element.

### Claim 7 (depends on 1)

The method of claim 1, wherein generating the text embedding comprises:
performing a diffusion process on the noise input.

### Claim 8 (depends on 1)

The method of claim 1, further comprising:
generating a synthetic image based on the text prompt, wherein the synthetic image includes the first element and the second element.

### Claim 9 (depends on 1)

The method of claim 1, wherein:
the embedding generation model is trained using training data including a ground truth text prompt describing a plurality of elements and a training image with a mask obscuring at least one of the plurality of elements.

### Claim 10

A method for training a machine learning model, the method comprising:
obtaining training data including a ground-truth text prompt describing a plurality of elements and a training image with a mask obscuring at least one of the plurality of elements; and
training, using the training data, an embedding generation model to generate a text embedding that represents the plurality of elements.

### Claim 11 (depends on 10)

The method of claim 10, wherein obtaining the training data comprises:
generating the ground-truth text prompt and the training image based on a common source image.

### Claim 12 (depends on 11)

The method of claim 11, wherein generating the training image comprises:
segmenting the common source image, wherein the mask is based on the segmentation.

### Claim 13 (depends on 10)

The method of claim 10, wherein training the embedding generation model comprises:
computing a diffusion loss based on the ground-truth text prompt; and
updating parameters of the embedding generation model based on the diffusion loss.

### Claim 14 (depends on 10)

The method of claim 10, further comprising:
training a text decoder to generate a text prompt based on the text embedding.

### Claim 15 (depends on 14)

The method of claim 14, wherein training the text decoder comprises:
computing a cross-entropy loss based on the ground truth text prompt; and
updating parameters of the text decoder based on the cross-entropy loss.

### Claim 16

An apparatus comprising:
at least one processor;
at least one memory storing instructions executable by the at least one processor;
the apparatus further comprising an embedding generation model comprising parameters stored in the at least one memory and trained to generate a text embedding from an image depicting a first element, wherein the text embedding represents the first element and a second element determined by the embedding generation model; and
a text decoder comprising parameters stored in the at least one memory and trained to generate a text prompt based on the text embedding.

### Claim 17 (depends on 16)

The apparatus of claim 16, further comprising:
an image encoder configured to encode the image to obtain an image embedding.

### Claim 18 (depends on 16)

The apparatus of claim 16, further comprising:
an image generation model configured to generate a synthetic image based on the text prompt.

### Claim 19 (depends on 16)

The apparatus of claim 16, further comprising:
a segmentation component configured to generate a mask that obscures a portion of the image corresponding to the second element.

### Claim 20 (depends on 16)

The apparatus of claim 16, wherein:
the embedding generation model comprises a diffusion model.
