# US 12,108,988 B2 · System and method for measuring pupillary distance and uses thereof > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,108,988 B2 - Document ID: US12108988B2 - Status: Granted - Kind: B2 - Application number: 17/345,876 - Filed: 2021-06-11 - Published: 2021-12-23 - Granted: 2024-10-08 - Earliest claimed: 2020-06-17 - Assignees: Warby Parker Inc. - Applicants: Warby Parker Inc. - Inventors: Dee Celeste Goldberg; Benjamin Cohen; Taylor Alexandra Duffy - Web page: https://patentgrep.com/patent/US12108988B2 ## CPC - A61B 3/11: for measuring interpupillary distance or diameter of pupils - A61B 3/111: {for measuring interpupillary distance} - A61B 3/113: for determining or recording eye movement - A61B 3/14: Arrangements specially adapted for eye photography - G06T 7/60: Analysis of geometric attributes - G06T 17/20: Finite element generation, e.g. wire-frame surface description, {tesselation} - G06T 2200/24: involving graphical user interfaces [GUIs] - G06T 2207/10028: Range image; Depth image; 3D point clouds - G06T 2207/30041: Eye; Retina; Ophthalmic - G06T 2207/30201: Face - G06T 2210/41: Medical - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A method of operating a pupillary distance system is disclosed. The method comprises the steps of capturing, with at least one camera of the pupillary distance system a 2D image and a corresponding 3D depth map of a face of a subject. A determination of pupil localization information is made using the 2D image and corresponding 3D depth map. The pupil location is further refined based on the pupil localization information. Pupil center coordinates are determined and the pupillary distance is calculated for a subject between centers of each pupil. Processes and uses thereof are also disclosed. ## Claims ### Claim 1 A method of operating a pupillary distance (“PD”) system, the method comprising the steps of: capturing, with at least one camera of the pupillary distance system, a first 2D image and a corresponding 3D depth map of a face of a subject; determining pupil localization information using the first 2D image and corresponding 3D depth map; refining one or more pupil locations based on the pupil localization information using a convolution with one or more kernels and one or more 2D center-surround filters; determining one or more pupil center coordinates; and calculating the PD of the subject between centers of each pupil. ### Claim 2 (depends on 1) The method of claim 1, wherein the pupil localization information is determined using a plurality of face mesh landmarks to generate a plurality of face mesh vertices near the center of an eye opening to obtain one or more initial pupil locations on the subject. ### Claim 3 (depends on 1) The method of claim 1, wherein the one or more kernels use a pupil estimate size of approximately 12 mm. ### Claim 4 (depends on 1) The method of claim 1, wherein calculating the PD uses depth map values that correspond to the refined one or more pupil locations in the 2D image. ### Claim 5 (depends on 1) The method of claim 1, wherein calculating the PD is determined by using points on a 3D face mesh that correspond to the centers of each pupil. ### Claim 6 (depends on 1) The method of claim 1, further comprising the steps of: performing a correction on the PD calculated using a distance that the first 2D image is taken from the at least one camera. ### Claim 7 A non-transitory computer readable medium having computer-executable instructions embodied thereon, wherein, when executed by a processor, the computer-executable instructions cause the processor to: obtain, from at least one camera, a first 2D image and a corresponding 3D depth map of a face of a subject; determine pupil localization information using the first 2D image and corresponding 3D depth map; refine one or more pupil locations based on the pupil localization information using a convolution with one or more kernels and one or more 2D center-surround filters; determine one or more pupil center coordinates; and calculate the PD of the subject between centers of each pupil. ### Claim 8 (depends on 7) The non-transitory computer readable medium of claim 7, wherein the pupil localization information is determined using a plurality of face mesh landmarks to generate a plurality of face mesh vertices near the center of an eye opening to obtain one or more initial pupil locations of the subject. ### Claim 9 (depends on 7) The non-transitory computer readable medium of claim 7, wherein the one or more kernels use a pupil estimate size of approximately 12 mm. ### Claim 10 (depends on 7) The non-transitory computer readable medium of claim 7, wherein calculating the PD uses depth map values that correspond to the refined one or more pupil locations in the 2D image. ### Claim 11 (depends on 7) The non-transitory computer readable medium of claim 7, wherein calculating the PD is determined by using points on a 3D face mesh that correspond to the centers of each pupil. ### Claim 12 (depends on 7) The non-transitory computer readable medium of claim 7, wherein the computer-executable instructions further cause the processor to perform a correction on the PD calculated using a distance that the first 2D image is taken from the at least one camera. ### Claim 13 A pupillary distance (“PD”) system, comprising: a mobile device, comprising: at least one camera; memory storing information associated with images and information obtained from the at least one camera; and a processor configured to: obtain, from the at least one camera, a 2D image and a corresponding 3D depth map of a face of a subject; determine pupil localization information using the 2D image and corresponding 3D depth map; refine one or more pupil locations based on the pupil localization information using a convolution with one or more kernels and one or more 2D center-surround filters; determine one or more pupil center coordinates; and calculate the PD between centers of each pupil of the subject. ### Claim 14 (depends on 13) The PD system of claim 13, wherein the pupil localization is determined using a plurality of face mesh landmarks to generate a plurality of face mesh vertices near the center of an eye opening to obtain one or more initial pupil locations of the subject. ### Claim 15 (depends on 13) The PD system of claim 13, wherein the one or more kernels use a pupil estimate size of approximately 12 mm. ### Claim 16 (depends on 13) The PD system of claim 13, wherein calculating the PD uses depth map values that correspond to the refined one or more pupil locations in the 2D image. ### Claim 17 (depends on 13) The PD system of claim 13, wherein calculating the PD is determined by using points on a 3D face mesh that correspond to the centers of each pupil.