US 12,505,629 B1Granted

Model-based augmented image generation

AssigneeAmazon Technologies, Inc.

InventorsOleksandr Vorobiov, Wei-Lin Hsiao, Nicolas Heron, Apoorv Chaudhri

Life of this patent

Estimating the term…

Loading patent…

Abstract

Techniques are described herein for generating a virtual try-on augmented image. An example method can include A system can access a first two-dimensional model of a target garment. The system can determine a first plurality of shape keypoints for a first part of the first plurality of parts of the image. The system can access a second two-dimensional image of a first user. The system can determine a second plurality of keypoints for a second part of the second plurality of parts of the second two-dimensional image, a second number of the second plurality of keypoints based at least in part on a second predetermined number of keypoints associated with the second part. The system can generate a rendered augmented image of the target garment on the target user.

Claims20

Dependent claims

A computer-implemented method, comprising:

accessing, by a computing system, a first two-dimensional image of a first user wearing a target garment, the first two-dimensional image comprising a first pixel having a first x-coordinate and a first y-coordinate;
generating, by the computing system, a first three-dimensional model of the first user based at least in part on the first two-dimensional image;
segmenting, by the computing system, the first two-dimensional image into a first plurality of parts;
determining, by the computing system, a first pose estimation of the first user;
determining, by the computing system, a first plurality of shape keypoints for a first part of the first plurality of parts, a first shape keypoint of the first plurality of shape keypoints arranged on the first part of the first plurality of parts based at least in part on the first pose estimation, the first shape keypoint represented by a first plurality of shape keypoint coordinates and corresponding to the first pixel;
accessing, by the computing system, a second two-dimensional image of a second user, the second two-dimensional image comprising a second pixel having a second x-coordinate and a second y-coordinate;
generating, by the computing system, a second three-dimensional model of the second user based at least in part on the second two-dimensional image;
segmenting, by the computing system, the second two-dimensional image into a second plurality of parts;
determining, by the computing system, a second pose estimation of the second user;
determining, by the computing system, a second part of the second plurality of parts based at least in part on the second pose estimation, the second part corresponding to the first part;
determining, by the computing system, a second plurality of shape keypoints for a second part of the second plurality of parts, a second shape keypoint of the second plurality of shape keypoints arranged on the second part of the second plurality of parts based at least in part on the second pose estimation, the second shape keypoint represented by a second plurality of shape keypoint coordinates, the second plurality of shape keypoint coordinates providing first physical characteristic of the target user;
and
generating, by the computing system, a rendered augmented image of the second user wearing the target garment the first plurality of shape keypoints and the second plurality of shape keypoint coordinates.

10 more elements

A computing system, comprising:

one or more processors;
and
one or more non-transitory computer-readable media comprising computer-executable instructions that, when executed by the one or more processors, cause the computing system to at least:
access a first two-dimensional image of a target garment;
access a second two-dimensional image of a first user;
segment the second two-dimensional image into a first plurality of parts;
determine a first plurality of shape keypoints for a first part of the first plurality of parts, a first number of the first plurality of shape keypoints based at least in part on a first predetermined number of shape keypoints associated with the first part;
and
generate a rendered augmented image of the first user wearing the target garment based at least in part on the first plurality of shape keypoints and the first two-dimensional image of the target garment.

6 more elements

One or more non-transitory computer-readable media comprising computer-executable instructions that, when executed by the one or more processors, cause a computing system to at least:

receive, at a first machine learning model of a computing system, a query vector generated based at least in part on a first shape keypoint of a first plurality of shape keypoints associated with a first user image, the first plurality of shape keypoints associated with an anatomical body part;
receive, at the machine learning model of the computing system, a key vector generated based at least in part on the first shape keypoint;
arrange, by the first machine learning model of the computing system, a pixel associated with the first shape keypoint on an image of first user using a cross-attention mechanism based at least in part on the query vector and the key vector, the pixel associated with a target garment image;
compare, by the computing system, an arrangement of the pixel to a ground truth arrangement of the pixel using a loss function;
and
adjust, by the computing system, a first weight of the first machine learning model based at least in part on the loss function.

3 more elements

Dependencies are inferred from the claim text.