Generating Images for Virtual Try-On and Pose Transfer

A system comprising:

• a candidate module implemented at least partially in hardware of a computing device to:
• receive input data describing a first digital image that depicts a person in a pose and a second digital image that depicts a garment;
• and
• compute candidate appearance flow maps that warp the garment based on the pose at different pixel-block sizes using a first convolutional network;
• a combination module implemented at least partially in the hardware of the computing device to generate a warped garment image by combining the candidate appearance flow maps as an aggregate per-pixel displacement map using a convolutional gated recurrent network, the aggregate per-pixel displacement map warps pixels depicting portions of the garment to align with the pose;
• a segment module implemented at least partially in the hardware of the computing device to predict a conditional segmentation mask that segments portions of a geometry of the person using a second convolutional network;
• and
• an output module implemented at least partially in the hardware of the computing device to output a digital image depicting the person in the pose wearing the garment based on the warped garment image and the conditional segmentation mask using a third convolutional network.

One or more computer-readable storage media comprising instructions stored thereon that, responsive to execution by a computing device, causes the computing device to perform operations including:

• receiving input data describing a first digital image that depicts a person in a pose wearing a first garment and a second digital image that depicts a second garment;
• computing candidate appearance flow maps that warp the second garment based on the pose at different pixel-block sizes using a first convolutional network;
• generating a warped garment image by combining the candidate appearance flow maps as an aggregate per-pixel displacement map using a convolutional gated recurrent network, the aggregate per-pixel displacement map warps pixels depicting portions of the second garment to align with the pose;
• predicting a conditional segmentation mask that segments portions of a geometry of the person using a second convolutional network;
• and
• outputting a digital image that depicts the person in the pose wearing the second garment based on the warped garment image and the conditional segmentation mask using a third convolutional network.