Real-time fashion item transfer system

A system comprising:

• at least one processor of a device;
• and
• a memory component having instructions stored thereon that, when executed by the at least one processor, cause the at least one processor to perform operations comprising:
• determining a pose of a three-dimensional (3D) avatar model;
• obtaining a first set of body landmarks corresponding to the 3D avatar model in the pose and a second set of body landmarks corresponding to a real-world object wearing a fashion item;
• computing a deviation between the first set of body landmarks and the second set of body landmarks;
• modifying the first set of body landmarks associated with the real-world object to match the second set of body landmarks associated with the 3D avatar model based on the deviation;
• applying a fitting model to the first and second sets of body landmarks to adjust one or more visual parameters of a graphic item corresponding to the modified first set of body landmarks;
• generating an intermediate image by the fitting model depicting the graphic item with the adjusted one or more visual parameters overlaid on the 3D avatar model;
• and
• applying a generative machine learning model to the intermediate image to blend sets of pixels corresponding to one or more gaps or occlusions that appear in the intermediate image.

A non-transitory computer-readable storage medium having stored thereon instructions that, when executed by at least one processor, cause the at least one processor to perform operations comprising:

• determining a pose of a three-dimensional (3D) avatar model;
• obtaining a first set of body landmarks corresponding to the 3D avatar model in the pose and a second set of body landmarks corresponding to a real-world object wearing a fashion item;
• computing a deviation between the first set of body landmarks and the second set of body landmarks;
• modifying the first set of body landmarks associated with the real-world object to match the second set of body landmarks associated with the 3D avatar model based on the deviation;
• applying a fitting model to the first and second sets of body landmarks to adjust one or more visual parameters of a graphic item corresponding to the modified first set of body landmarks;
• generating an intermediate image by the fitting model depicting the graphic item with the adjusted one or more visual parameters overlaid on the 3D avatar model;
• and
• applying a generative machine learning model to the intermediate image to blend sets of pixels corresponding to one or more gaps or occlusions that appear in the intermediate image.