# US 12,602,850 B2 · Generative AI virtual clothing try-on

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,602,850 B2
- Document ID: US12602850B2
- Status: Granted
- Kind: B2
- Application number: 18/369,935
- Filed: 2023-09-19
- Published: 2025-01-30
- Granted: 2026-04-14
- Earliest claimed: 2023-07-27
- Assignees: Snap Inc.
- Applicants: Snap Inc.
- Inventors: Mohammad Rami Koujan; Iason Kokkinos
- Web page: https://patentgrep.com/patent/US12602850B2

## CPC

- G06T 3/40: Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06T 7/90: Determination of colour characteristics
- G06T 11/60: Creating or editing images; Combining images with text
- G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI]
- G06V 10/82: using neural networks
- G06V 40/103: {Static body considered as a whole, e.g. static pedestrian or occupant recognition}
- G06T 2207/10024: Color image
- G06T 2207/20044: Skeletonization; Medial axis transform
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

An artificial intelligence (AI) network or neural network is trained, using a relatively small number of reference images of a target garment, to enable virtual clothing try-ons of the target garment. Example methods include determining a pose for a person depicted in an input image, determining an area of the input image to replace with a target garment, changing values of pixels within the area, and inputting the pose, the area, and a text prompt describing the target garment, into a neural network, to generate an output image, wherein the neural network is trained to generate the target garment. Example methods include training the neural network with images of clothing in a same class or category as the target garment to teach the neural network to shape the target garment in accordance with a pose of the person and to preserve other clothing and the background.

## Claims

### Claim 1

A computing device comprising:
one or more processors; and
one or more memories storing instructions that, when executed by the one or more processors, configure the one or more processors to perform operations comprising:
determining a pose for a person depicted in an input image;
determining body part masks for the input image;
determining a bounding box for the input image, the bounding box comprising the person depicted within the input image;
determining clothing masks for clothing to be preserved;
determining an area, of the input image to replace with a target garment, as the clothing masks and the body part masks subtracted from the bounding box;
changing values of pixels within the area;
inputting the pose, the area, and a text prompt describing the target garment, into a neural network, to generate an output image, wherein the neural network is trained to generate the target garment within the area; and
blending the body part masks with the output image.

### Claim 2 (depends on 1)

The computing device of claim 1, wherein the determining the pose for the person depicted in the input image comprises:
identifying key points of the person depicted in the input image; and
connecting the key points.

### Claim 3 (depends on 1)

The computing device of claim 1, wherein the blending is within the area.

### Claim 4 (depends on 1)

The computing device of claim 1, wherein the values of the pixels are changed to a same value or the values of the pixels are changed based on sampling a Gaussian distribution.

### Claim 5 (depends on 1)

The computing device of claim 1, wherein the neural network is a first neural network, and wherein determining the area comprises:
inputting the input image into a second neural network to generate the area.

### Claim 6 (depends on 5)

The computing device of claim 5, wherein the input image is a first input image, the person is a first person, the area is a first area, and wherein the second neural network is trained to identify clothing of a same category as the target garment.

### Claim 7 (depends on 1)

The computing device of claim 1, wherein the input image is a first input image, the person is a first person, the area is a first area, the output image is a first output image, and wherein the operations further comprise:
determining a second pose for a second person depicted in a second input image, the second person wearing the target garment;
determining a second area of the second input image;
changing values of pixels within the second area of the second input image;
inputting the second pose, the second area, and the text prompt describing the target garment, into the neural network, to generate a second output image;
determining losses between the second output image and the second input image; and
updating weights of the neural network based on the losses.

### Claim 8 (depends on 7)

The computing device of claim 7, wherein the operations further comprise:
determining a detailed area of the target garment; and
enlarging the detailed area of the target garment within the input image.

### Claim 9 (depends on 7)

The computing device of claim 7, wherein the losses are first losses, the text prompt is a first text prompt, and wherein the operations further comprise:
determining third poses of people depicted in a plurality of third input images, each of the people wearing a garment of a garment category of the target garment;
changing values of pixels within third areas of the plurality of third input images;
inputting the third poses, the third areas, and a second text prompt describing the people and the garment category, into the neural network, to generate a plurality of third output images;
determining second losses between the plurality of third output images and the plurality of third input images; and
updating the weights of the neural network based on the second losses.

### Claim 10 (depends on 9)

The computing device of claim 9, wherein the operations further comprise:
freezing the weights of some blocks of a non-pose portion of the neural network.

### Claim 11 (depends on 1)

The computing device of claim 1, wherein the text prompt further describes a characteristic of the person depicted in the input image.

### Claim 12 (depends on 11)

The computing device of claim 11, wherein the operations further comprise:
determining the characteristic of the person, wherein the characteristic comprises at least one of a gender of the person, a skin tone of the person, a hair color of the person, an indication if gloves are worn by the person, an indication if glasses are worn by the person, an indication of how much of a body of the person is depicted in the image, and an indication of a color of shoes worn by the person.

### Claim 13 (depends on 1)

The computing device of claim 1, wherein the output image is a first output image, and wherein the operations further comprise:
changing values of pixels outside the area; and
inputting the pose, the area, and the text prompt describing the target garment, into the neural network, to generate a second output image.

### Claim 14 (depends on 1)

The computing device of claim 1, wherein the output image is a first output image, and wherein the operations further comprise:
changing values of pixels within the area of the first output image; and
inputting the pose, the first output image, and the text prompt describing the target garment, into the neural network, to generate a second output image.

### Claim 15

A non-transitory computer-readable storage medium including instructions that, when processed by one or more processors, configure the one or more processors to perform operations comprising:
determining a pose for a person depicted in an input image;
determining body part masks for the input image;
determining a bounding box for the input image, the bounding box comprising the person depicted within the input image;
determining clothing masks for clothing to be preserved;
determining an area, of the input image to replace with a target garment, as the clothing masks and the body part masks subtracted from the bounding box;
changing values of pixels within the area;
inputting the pose, the area, and a text prompt describing the target garment, into a neural network, to generate an output image, wherein the neural network is trained to generate the target garment within the area; and
blending the body part masks with the output image.

### Claim 16 (depends on 15)

The non-transitory computer-readable storage medium of claim 15, wherein the determining the pose for the person depicted in the input image comprises:
identifying key points of the person depicted in the input image; and
connecting the key points.

### Claim 17 (depends on 15)

The non-transitory computer-readable storage medium of claim 15, wherein the input image is a first input image, the person is a first person, the area is a first area, the output image is a first output image, and wherein the operations further comprise:
determining a second pose for a second person depicted in a second input image, the second person wearing the target garment;
determining a second area of the second input image;
changing values of pixels within the second area of the second input image;
inputting the second pose, the second area, and the text prompt describing the target garment, into the neural network, to generate a second output image;
determining losses between the second output image and the second input image; and
updating weights of the neural network based on the losses.

### Claim 18

A method comprising:
determining a pose for a person depicted in an input image;
determining body part masks for the input image;
determining a bounding box for the input image, the bounding box comprising the person depicted within the input image;
determining clothing masks for clothing to be preserved;
determining an area, of the input image to replace with a target garment, as the clothing masks and the body part masks subtracted from the bounding box;
changing values of pixels within the area;
inputting the pose, the area, and a text prompt describing the target garment, into a neural network, to generate an output image, wherein the neural network is trained to generate the target garment within the area; and
blending the body part masks with the output image.

### Claim 19 (depends on 18)

The method of claim 18, wherein the determining the pose for the person depicted in the input image comprises:
identifying key points of the person depicted in the input image; and
connecting the key points.

### Claim 20 (depends on 18)

The method of claim 18, wherein the input image is a first input image, the person is a first person, the area is a first area, the output image is a first output image, and wherein the method further comprises:
determining a second pose for a second person depicted in a second input image, the second person wearing the target garment;
determining a second area of the second input image;
changing values of pixels within the second area of the second input image;
inputting the second pose, the second area, and the text prompt describing the target garment, into the neural network, to generate a second output image;
determining losses between the second output image and the second input image; and
updating weights of the neural network based on the losses.