Eyewear distortion correction
AssigneeSnap Inc.
InventorsEyal ZAK, Guy MELAMED
Life of this patent
Estimating the term…
Abstract
Systems, methods, and non-transitory computer readable mediums including processes to correct for distortion in images introduced by eyewear (i.e., where the facial region surrounding the eye has a boundary that doesn't match the boundary of an uncovered facial region). The correction includes segmenting images to detect eyewear covered facial regions and facial regions not covered by the eyewear and altering the covered facial regions to match the covered facial boundary to the uncovered facial boundary. Alterations include processing using a machine learning model, applying anti-refraction algorithms, scaling the covered facial region to match boundaries of the uncovered facial region, or a combination thereof.
Claims24
A method for addressing facial distortion in images due to eyewear, the method comprising:
- obtaining an image including a face of a subject wearing eyewear, the eyewear including a lens defining a coverage area covering a region of the face, the lens distorting the region of the face in the coverage area to produce a covered facial region within the coverage area of the lens, the covered facial region having a covered facial boundary that is not aligned with an uncovered facial boundary of the face outside the coverage area;
- segmenting the obtained image to detect the face and the coverage area of the lens;
- altering the covered facial region within the detected coverage area of the lens to match the covered facial boundary to the uncovered facial boundary by identifying an interior upper edge of the covered facial boundary where the covered facial boundary intersects an upper edge of the lens, identifying an interior lower edge of the covered facial boundary where the covered facial boundary intersects a lower edge of the lens, identifying an exterior upper edge of the uncovered facial boundary where the uncovered facial boundary interests an upper edge of the lens, identifying an exterior lower edge of the uncovered facial boundary where the uncovered facial boundary intersects a lower edge of the lens, and scaling the covered facial boundary in a horizontal direction until at least one of the interior upper edge matches the exterior upper edge or the interior lower edge matches the exterior lower edge;
- and
- displaying the obtained image with the altered covered facial region.
(canceled)
(canceled)
(canceled)
A system for addressing facial distortion in images due to eyewear, the system comprising:
- an image capture device configured to obtain an image including a face of a subject wearing eyewear, the eyewear including a lens defining a coverage area covering a region of the face, the lens distorting the region of the face in the coverage area to produce a covered facial region within the coverage area of the lens, the covered facial region having a covered facial boundary that is not aligned with an uncovered facial boundary of the face outside the coverage area;
- a processor coupled to the image capture device, the processor configured to segment the obtained image to detect the face and the coverage area of the lens and to alter the covered facial region within the detected coverage area of the lens to match the covered facial boundary to the uncovered facial boundary by being configured to identify an interior upper edge of the covered facial boundary where the covered facial boundary intersects an upper edge of the lens, identify an interior lower edge of the covered facial boundary where the covered facial boundary intersects a lower edge of the lens, identify an exterior upper edge of the uncovered facial boundary where the uncovered facial boundary interests an upper edge of the lens, identify an exterior lower edge of the uncovered facial boundary where the uncovered facial boundary intersects a lower edge of the lens, and scale the covered facial boundary in a horizontal direction until at least one of the interior upper edge matches the exterior upper edge or the interior lower edge matches the exterior lower edge;
- and
- a display coupled to the processor, the display configured to display the obtained image with the altered covered facial region.
(canceled)
(canceled)
(canceled)
A non-transitory computer-readable medium for addressing facial distortion in images due to eyewear, the non-transitory computer-readable medium comprising instructions that, when performed by a processor, configure the processor to performed functions, including functions to:
- obtain an image including a face of a subject wearing eyewear, the eyewear including a lens defining a coverage area covering a region of the face, the lens distorting the region of the face in the coverage area to produce a covered facial region within the coverage area of the lens, the covered facial region having a covered facial boundary that is not aligned with an uncovered facial boundary of the face outside the coverage area;
- segment the obtained image to detect the face and the coverage area of the lens;
- alter the covered facial region within the detected coverage area of the lens to match the covered facial boundary to the uncovered facial boundary by configuring the processor to identify an interior upper edge of the covered facial boundary where the covered facial boundary intersects an upper edge of the lens, identify an interior lower edge of the covered facial boundary where the covered facial boundary intersects a lower edge of the lens, identify an exterior upper edge of the uncovered facial boundary where the uncovered facial boundary interests an upper edge of the lens, identify an exterior lower edge of the uncovered facial boundary where the uncovered facial boundary intersects a lower edge of the lens, and scale the covered facial boundary in a horizontal direction until at least one of the interior upper edge matches the exterior upper edge or the interior lower edge matches the exterior lower edge;
- and
- display the obtained image with the altered covered facial region.
(canceled)
(canceled)
(canceled)
Dependencies are inferred from the claim text.