US 11,080,817 B2 · Cloth warping using multi-scale patch adversarial loss

Abstract

Generating a synthesized image of a person wearing clothing is described. A two-dimensional reference image depicting a person wearing an article of clothing and a two-dimensional image of target clothing in which the person is to be depicted as wearing are received. To generate the synthesized image, a warped image of the target clothing is generated via a geometric matching module, which implements a machine learning model trained to recognize similarities between warped and non-warped clothing images using multi-scale patch adversarial loss. The multi-scale patch adversarial loss is determined by sampling patches of different sizes from corresponding locations of warped and non-warped clothing images. The synthesized image is generated on a per-person basis, such that the target clothing fits the particular body shape, pose, and unique characteristics of the person.

Claims20

Dependent claims

In a digital medium environment for generating a synthesized image of a person wearing target clothing, a method implemented by at least one computing device, the method comprising:

receiving, by the at least one computing device, a reference image depicting a person wearing an article of clothing;
receiving, by the at least one computing device, an image of target clothing in which the person is to be depicted as wearing;
generating, by the at least one computing device, a warped image of the target clothing using a geometric matching module trained using multi-scale patch adversarial loss, the warped image of the target clothing being generated based on a person representation derived from the reference image and the multi-scale patch adversarial loss being computed using a plurality of pairs of patches sampled from corresponding locations in a mask of the article of clothing derived from the reference image and a product image depicting the article of clothing without being worn by a person;
and
generating, by the at least one computing device, a synthesized image that depicts the person wearing the warped image of the target clothing instead of the article of clothing.

2 more elements

In a digital medium environment for training a geometric matching module using multi-scale patch adversarial loss, a method implemented by at least one computing device, the method comprising:

receiving, by the at least one computing device, a plurality of image pairs, each image pair including a reference image that depicts a person wearing an article of clothing and a product image that depicts the article of clothing not being worn by a person;
for each of the plurality of image pairs:
determining, by the at least one computing device, a clothing-agnostic person representation for the person wearing the article of clothing;
generating, by the at least one computing device, a warped version of the article of clothing;
determining, by the at least one computing device, a ground truth warped cloth for the article of clothing;
and
sampling, by the at least one computing device, a plurality of pairs of patches from the warped version of the article of clothing and the ground truth warped cloth;
and
training, by the at least one computing device, the geometric matching module to output a warped cloth for a target piece of clothing using multi-scale patch adversarial loss, the multi-scale patch adversarial loss determined by:
providing a pair of patches from the plurality of pairs of patches sampled from same corresponding regions of a warped version of the article of clothing and the ground truth warped cloth as positive training examples;
and
providing a pair of patches from the plurality of pairs of patches sampled from different regions of the warped version of the article of clothing and the ground truth warped cloth as negative training examples.

9 more elements

Dependencies are inferred from the claim text.

Cloth warping using multi-scale patch adversarial loss

Life of this patent

Cloth warping using multi-scale patch adversarial loss

Life of this patent

Abstract

Claims20