US 10,540,757 B1Granted

Method and system for generating combined images utilizing image processing of multiple images

AssigneeAmazon Technologies, Inc.

InventorsMoshe Bouhnik, Hilit Unger, Eduard Oks, Noam Sorek

Life of this patent

Estimating the term…

Loading patent…

Abstract

A computer-implemented method includes receiving first pose data for a first human represented in a first image, receiving second pose data for a second human represented in a second image, receiving first semantic segmentation data for the first image, and receiving second semantic segmentation data for the second image. A pose-aligned second image can be generated by modifying the second image based on the first pose data, the second pose data, the first semantic segmentation data, and the second semantic segmentation data. A mixed image can be determined by combining pixel values from the first image and pixel values of the pose-aligned second image according to mask data. In some embodiments, the mixed image includes a representation of an outfit that includes first clothing represented in the first image and second clothing represented in the second image.

Claims20

Dependent claims

A computer-implemented method, comprising:

obtaining a first image that represents a first person wearing a shirt and a skirt;
obtaining a second image that represents a second person wearing a top and pants;
generating a mixed image that represents the first person wearing the shirt and the pants, wherein generating the mixed image includes:
determining first semantic segmentation data for the first image, wherein the first semantic segmentation data identifies pixel locations for at least a head, arms, hands, and the shirt of the first person;
determining second semantic segmentation data for the second image, wherein the second semantic segmentation data identifies pixel locations for at least the pants of the second person;
determining a pose-aligned second image;
determining mask data using the first image, the pose-aligned second image, and the first and second semantic segmentation data;
and
determining pixel values of the mixed image by combining pixel values from the first image with pixel values of the pose-aligned second image based on the mask data;
wherein determining the pose-aligned second image includes:
determining, using pose data, a first pose of the first person and a second pose of the second person;
determining a scaled second image by scaling the second image such that a height of the second person substantially equals a height of the first person, wherein the height of the first person and the height of the second person are determined using the first and second semantic segmentation data, respectively;
and
rotating and translating the scaled second image to align the second pose to the first pose, and
wherein determining the mask data comprises:
based on the first semantic segmentation data, assigning a value of 1 to mask pixels that correspond to the head, arms, hands, and shirt of the first person;
and
based on the second semantic segmentation data, assigning a value of 0 to mask pixels that correspond to the pants of the second person.

15 more elements

A computer-implemented method, comprising:

receiving first pose data for a first human represented in a first image;
receiving second pose data for a second human represented in a second image;
receiving first semantic segmentation data for the first image;
receiving second semantic segmentation data for the second image;
generating a pose-aligned second image by modifying the second image based on the first pose data, the second pose data, the first semantic segmentation data, and the second semantic segmentation data;
generating mask data, using the first and second semantic segmentation data, wherein the mask data defines a set of conditions for combining pixels of the first image with pixels of the pose-aligned second image;
and
determining a mixed image by combining pixel values from the first image and pixel values of the pose-aligned second image according to the mask data, wherein the mixed image includes a representation of an outfit that includes first clothing represented in the first image and second clothing represented in the second image.

5 more elements

Dependencies are inferred from the claim text.