# US 2025/0166236 A1 · Segmentation free guidance in diffusion models

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0166236 A1
- Document ID: US20250166236A1
- Status: Published application
- Kind: A1
- Application number: 18/511,692
- Filed: 2023-11-16
- Published: 2025-05-22
- Earliest claimed: 2023-11-16
- Assignees: QUALCOMM Incorporated
- Applicants: QUALCOMM Incorporated
- Inventors: Kambiz AZARIAN YAZDI; Fatih Murat PORIKLI; Qiqi HOU; Debasmit DAS
- Web page: https://patentgrep.com/patent/US20250166236A1

## CPC

- G06T 11/00: Two-dimensional [2D] image generation
- G06F 40/284: Lexical analysis, e.g. tokenisation or collocates
- G06T 5/70: Denoising; Smoothing
- G06T 2200/24: involving graphical user interfaces [GUIs]
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Certain aspects of the present disclosure provide techniques for generating an output image based on a text prompt. A method may include receiving the text prompt; providing a user interface comprising one or more input elements associated with one or more words of the text prompt; receiving input corresponding to at least one of the one or more input elements, the input indicating a semantic importance for each of at least one of the one or more words associated with the at least one of the one or more input elements; and generating the output image based on the text prompt and the input.

## Claims

### Claim 1

An apparatus configured to generate an output image based on a text prompt, comprising:
one or more memories configured to store a latent image representation; and
one or more processors, coupled to the one or more memories, configured to:
obtain the text prompt;
encode the text prompt into a plurality of conditioning tokens;
for each of one or more patches of the latent image representation:
calculate a respective plurality of cross-attention weights corresponding to the plurality of conditioning tokens based on the patch as a query and the plurality of conditioning tokens as a key; and
modify a maximum value cross-attention weight among the respective plurality of cross-attention weights to generate a modified respective plurality of cross-attention weights;
perform an iteration of denoising using the modified respective plurality of cross-attention weights for each of the one or more patches to obtain a modified latent image representation; and
generate the output image based on the modified latent image representation.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein to modify the maximum value cross-attention weight among the respective plurality of cross-attention weights comprises to reduce the maximum value cross-attention weight.

### Claim 3 (depends on 1)

The apparatus of claim 1, wherein to modify the maximum value cross-attention weight among the respective plurality of cross-attention weights comprises to multiply the maximum value cross-attention weight by a negative scalar value.

### Claim 4 (depends on 3)

The apparatus of claim 3, wherein the one or more processors are configured to receive the negative scalar value as a user-specified parameter.

### Claim 5 (depends on 1)

The apparatus of claim 1, wherein to modify the maximum value cross-attention weight among the respective plurality of cross-attention weights comprises to set the maximum value cross-attention weight to zero.

### Claim 6 (depends on 1)

The apparatus of claim 1, wherein the one or more processors are configured to perform one or more initial denoising iterations using classifier-free guidance prior to performing the iteration of denoising using the modified respective plurality of cross-attention weights for each of the one or more patches.

### Claim 7 (depends on 1)

The apparatus of claim 1, further comprising a display, coupled to the one or more processors, configured to display a user interface configured to receive input indicative of an emphasis strength associated with one or more words of the text prompt, wherein the emphasis strength controls an amount to modify the maximum value cross-attention weight among the respective plurality of cross-attention weights.

### Claim 8 (depends on 7)

The apparatus of claim 7, wherein the user interface includes one or more interface elements configured to receive the input, the one or more interface elements including at least one of a slider, a numerical input, or a keyword highlight.

### Claim 9 (depends on 1)

The apparatus of claim 1, wherein the one or more processors are configured to:
for each of one or more patches of the modified latent image representation:
calculate a respective second plurality of cross-attention weights corresponding to the plurality of conditioning tokens based on the patch as another query and the plurality of conditioning tokens as another key; and
modify another maximum value cross-attention weight among the respective second plurality of cross-attention weights to generate another modified respective plurality of second cross-attention weights; and
perform a second iteration of denoising using the modified respective plurality of second cross-attention weights for each of the one or more patches of the modified latent image representation to obtain a second modified latent image representation, wherein to generate the output image based on the modified latent image representation comprises to decode the second modified latent image representation using a decoder to generate the output image.

### Claim 10 (depends on 1)

The apparatus of claim 1, further comprising a display, coupled to the one or more processors, configured to display the output image.

### Claim 11 (depends on 1)

The apparatus of claim 1, further comprising:
a modem, coupled to the one or more processors, configured to modulate one or more carrier wave signals with data indicative of the output image.

### Claim 12 (depends on 11)

The apparatus of claim 11, further comprising:
one or more antennas, coupled to the modem, configured to transmit the one or more carrier wave signals to a device.

### Claim 13 (depends on 1)

The apparatus of claim 1, wherein to perform the iteration of denoising, the one or more processors are configured to:
generate a first output based on the latent image representation and the respective plurality of cross-attention weights for each of the one or more patches of the latent image representation;
generate a second output based on the latent image representation and the modified respective plurality of cross-attention weights for each of the one or more patches of the latent image representation; and
subtract the second output from the first output to obtain the modified latent image representation.

### Claim 14 (depends on 1)

The apparatus of claim 1, wherein to perform the iteration of denoising, the one or more processors are configured to:
generate the modified latent image representation based on the latent image representation and the modified respective plurality of cross-attention weights for each of the one or more patches of the latent image representation.

### Claim 15

An apparatus configured to generate an output image based on a text prompt, comprising:
one or more memories configured to store the output image;
one or more processors, coupled to the one or more memories, configured to:
receive the text prompt;
provide a user interface comprising one or more input elements associated with one or more words of the text prompt;
receive input corresponding to at least one of the one or more input elements, the input indicating a semantic importance for each of at least one of the one or more words associated with the at least one of the one or more input elements; and
generate the output image based on the text prompt and the input.

### Claim 16 (depends on 15)

The apparatus of claim 15, further comprising a display, coupled to the one or more processors, configured to:
display the user interface; and
display the output image.

### Claim 17 (depends on 15)

The apparatus of claim 15, wherein to generate the output image, the one or more processors are configured to generate the output image using the text prompt and the input as inputs to a generative artificial intelligence (AI) model.

### Claim 18 (depends on 15)

The apparatus of claim 15, wherein a first input element of the one or more input elements is a slider element configured to increase or decrease the importance of a first word of the one or more words.

### Claim 19 (depends on 15)

The apparatus of claim 15, wherein a first input element of the one or more input elements is a dial element configured to increase or decrease the importance of a first word of the one or more words.

### Claim 20 (depends on 15)

The apparatus of claim 15, wherein the one or more processors are configured to modify an appearance of the at least one of the one or more words based on the indicated semantic importance.

### Claim 21 (depends on 20)

The apparatus of claim 20, wherein to modify the appearance of the at least one of the one or more words comprises to highlight the at least one of the one or more words.

### Claim 22 (depends on 15)

The apparatus of claim 15, wherein the output image emphasizes one or more objects associated with the at least one of the one or more words indicated as having higher semantic importance as compared to other words of the one or more words.

### Claim 23 (depends on 15)

The apparatus of claim 15, further comprising a display, coupled to the one or more processors, configured to:
prior to the one or more processors receiving the input, display a first image associated with the text prompt; and
after the one or more processors receiving the input, display the output image.

### Claim 24 (depends on 23)

The apparatus of claim 23, wherein the display is configured to:
display a transition from the first image to the output image.

### Claim 25 (depends on 23)

The apparatus of claim 23, wherein the output image emphasizes one or more objects associated with the at least one of the one or more words as compared to the one or more objects in the first image.

### Claim 26 (depends on 23)

The apparatus of claim 23, wherein:
the one or more processors are configured to generate the first image using the text prompt as input to a generative artificial intelligence (AI) model; and
to generate the output image, the one or more processors are configured to generate the output image using the text prompt and the input as inputs to the generative AI model.

### Claim 27 (depends on 15)

The apparatus of claim 15, wherein the one or more processors are configured to:
encode the text prompt into a plurality of conditioning tokens;
for each of one or more patches of a latent image representation:
calculate a respective plurality of cross-attention weights corresponding to the plurality of conditioning tokens based on the patch as a query and the plurality of conditioning tokens as a key; and
modify a maximum value cross-attention weight among the respective plurality of cross-attention weights to generate a modified respective plurality of cross-attention weights; and
perform an iteration of denoising using the modified respective plurality of cross-attention weights for each of the one or more patches to generate the output image.

### Claim 28 (depends on 27)

The apparatus of claim 27, wherein at least one cross-attention weight among the respective plurality of cross-attention weights is modified based on the indicated semantic importance.

### Claim 29

A method comprising:
obtaining a text prompt;
encoding the text prompt into a plurality of conditioning tokens;
for each of one or more patches of a latent image representation:
calculating a respective plurality of cross-attention weights corresponding to the plurality of conditioning tokens based on the patch as a query and the plurality of conditioning tokens as a key; and
modifying a maximum value cross-attention weight among the respective plurality of cross-attention weights to generate a modified respective plurality of cross-attention weights;
performing an iteration of denoising using the modified respective plurality of cross-attention weights for each of the one or more patches to obtain a modified latent image representation; and
generating an output image based on the modified latent representation.

### Claim 30

A method comprising:
receiving a text prompt;
providing a user interface comprising one or more input elements associated with one or more words of the text prompt;
receiving input corresponding to at least one of the one or more input elements, the input indicating a semantic importance for each of at least one of the one or more words associated with the at least one of the one or more input elements; and
generating an output image based on the text prompt and the input.
