Back to results
US 2021/0201544 A1Published application
Face swap method and computing device
AssigneePing An Technology (Shenzhen) Co., Ltd.
InventorsJinghong Miao, Yuchuan Gou, Minghao Li, Jui-Hsin Lai, Bo Gong, Mei Han
Life of this patent
Estimating the term…
Abstract
In a face swap method carried out by an electronic device, a first head image is segmented from a destination image. First facial landmarks and a first hair mask are obtained according to the first head image. A second head image is segmented from a source image. Second facial landmarks and a second hair mask are obtained according to the second head image. If at least one eye landmark in the second facial landmarks is covered by hair, the second head image and the second hair mask are processed and repaired so as to obtain a swapped-face image with eyes not covered by hair.
Claims20
A face swap method being executed by a computing device, the method comprising:
- segmenting a first head image from a destination image;
- detecting first facial landmarks in the first head image;
- detecting a first hair region in the first head image, and obtaining a first hair mask according to the first hair region;
- segmenting a second head image from a source image;
- detecting second facial landmarks in the second head image;
- detecting a second hair region in the second head image, and obtaining a second hair mask according to the second hair region;
- determining whether any eye landmark in the second facial landmarks is covered by hair according to the second hair mask;
- in response that at least one eye landmark in the second facial landmarks is covered by hair, repairing the second head image and the second hair mask according to mirror landmarks of the at least one eye landmark to obtain a second repaired head image and a second repaired hair mask;
- segmenting a second facial image from the second repaired head image according to the second repaired hair mask;
- synthesizing the second facial image on the first head image according to the first facial landmarks to obtain a first synthetic image;
- covering the first hair region to the first synthetic image according to the first hair mask to obtain a second synthetic image;
- and
- filling the second synthetic image back to segmented destination image to obtain a face swap image.
10 more elements
A computing device comprising:
- a storage device;
- at least one processor;
- and
- the storage device storing one or more programs, which when executed by the at least one processor, cause the at least one processor to perform operations comprising:
- segmenting a first head image from a destination image;
- detecting first facial landmarks in the first head image;
- detecting a first hair region in the first head image, and obtaining a first hair mask according to the first hair region;
- segmenting a second head image from a source image;
- detecting second facial landmarks in the second head image;
- detecting a second hair region in the second head image, and obtaining a second hair mask according to the second hair region;
- determining whether any eye landmark in the second facial landmarks is covered by hair according to the second hair mask;
- in response that at least one eye landmark in the second facial landmarks is covered by hair, repairing the second head image and the second hair mask according to mirror landmarks of the at least one eye landmark to obtain a second repaired head image and a second repaired hair mask;
- segmenting a second facial image from the second repaired head image according to the second repaired hair mask;
- synthesizing the second facial image on the first head image according to the first facial landmarks to obtain a first synthetic image;
- covering the first hair region to the first synthetic image according to the first hair mask to obtain a second synthetic image;
- and
- filling the second synthetic image back to segmented destination image to obtain a face swap image.
14 more elements
A non-transitory storage medium having stored thereon instructions that, when executed by a computing device of a computing device, causes the computing device to perform a face swap method, the method comprising:
- segmenting a first head image from a destination image;
- detecting first facial landmarks in the first head image;
- detecting a first hair region in the first head image, and obtaining a first hair mask according to the first hair region;
- segmenting a second head image from a source image;
- detecting second facial landmarks in the second head image;
- detecting a second hair region in the second head image, and obtaining a second hair mask according to the second hair region;
- determining whether any eye landmark in the second facial landmarks is covered by hair according to the second hair mask;
- in response that at least one eye landmark in the second facial landmarks is covered by hair, repairing the second head image and the second hair mask according to mirror landmarks of the at least one eye landmark to obtain a second repaired head image and a second repaired hair mask;
- segmenting a second facial image from the second repaired head image according to the second repaired hair mask;
- synthesizing the second facial image on the first head image according to the first facial landmarks to obtain a first synthetic image;
- covering the first hair region to the first synthetic image according to the first hair mask to obtain a second synthetic image;
- and
- filling the second synthetic image back to segmented destination image to obtain a face swap image.
10 more elements
Dependencies are inferred from the claim text.