# US 12,008,711 B2 · Determining display gazability and placement of virtual try-on glasses using optometric measurements > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,008,711 B2 - Document ID: US12008711B2 - Status: Granted - Kind: B2 - Application number: 17/650,453 - Filed: 2022-02-09 - Published: 2023-08-10 - Granted: 2024-06-11 - Earliest claimed: 2022-02-09 - Assignees: Google Inc. - Applicants: Google LLC - Inventors: Idris Syed Aleem; Rees Anwyl Samuel Simmons - Web page: https://patentgrep.com/patent/US12008711B2 ## CPC - G02C 13/003: {Measuring during assembly or fitting of spectacles} - G06T 17/00: Three-dimensional [3D] modelling for computer graphics - G02C 13/005: {Measuring geometric parameters required to locate ophtalmic lenses in spectacles frames (apparatus for testing or instruments for examining the eyes per se A61B3/00; machines or devices and accessories for grinding the edges of lenses using spectacles as a template B24B9/144)} - G06Q 30/0621: {by configuring or customising goods or services} - G06Q 30/0643: {graphically representing goods, e.g. 3D product representation} - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G02B 2027/014: {comprising information/image processing systems} - G06T 2207/30201: Face - G02C: SPECTACLES; SUNGLASSES OR GOGGLES INSOFAR AS THEY HAVE THE SAME FEATURES AS SPECTACLES; CONTACT LENSES (trial frames for testing the eyes A61B3/04; goggles or eyeshields not having the same features as spectacles A61F9/00) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS ## Abstract A method includes receiving a face image and lens measurements of a person, and selecting a three dimensional (3-D) model of a frame of a pair of smart glasses from a plurality of 3-D models of frames for placement on the face image of the person. The pair of smart glasses includes a virtual display embedded in, or overlaid on a lens. The method further includes sizing the selected 3-D model of the frame of the pair of smart glasses for placement on the face image of person based on the lens measurements of the person, and determining a gazability of the virtual display in the 3-D model of the pair of smart glasses placed on the face image of the person. ## Claims ### Claim 1 An image processing system, comprising: a processor; a memory; and the processor being configured to execute instructions stored in memory to: receive a face image of a person; receive lens measurements of the person; select a model of a frame of a pair of smart glasses from a plurality of models of frames for placement on the face image of the person, the pair of smart glasses including a display embedded in, or overlaid on a lens of the pair of smart glasses; size the selected model of the frame of the pair of smart glasses for placement on the face image of the person based on the lens measurements; and determine a gazability measure of the display by adjusting a gaze of a pupil of the person to each corner of the display and determining how much light transmitted through the display enters the pupil. ### Claim 2 (depends on 1) The image processing system of claim 1, wherein the received lens measurements include one or more of a left pupillary distance, right pupillary distance, pupil height, pantoscopic angle, and vertex distance. ### Claim 3 (depends on 1) The image processing system of claim 1, wherein the received face image of the person is a frontal face image of the person. ### Claim 4 (depends on 1) The image processing system of claim 1, further comprising, a frames library including the plurality of the models of the frames, and wherein the selected model of the frame is selected from the frames library. ### Claim 5 (depends on 1) The image processing system of claim 1, wherein sizing the selected model of the frame of the pair of smart glasses for placement on the person includes placing the selected model of the frame in 3-D space on or aligned with a 3-D model of a head of the person. ### Claim 6 (depends on 5) The image processing system of claim 5, wherein placing the selected frame in 3-D space on or aligned with the 3-D model of the head of the person includes determining a location of a right pupil of the person by combining information available in the lens measurements including right pupillary distance (RPD), pupil height, and vertex distance. ### Claim 7 (depends on 1) The image processing system of claim 1, wherein the processor is configured to execute instructions stored in memory to further receive side-view face images of the person, and align and place the sized model of the frame of the pair of smart glasses on the side-view face images of the person. ### Claim 8 A method, comprising: receiving a face image of a person; receiving lens measurements of the person; selecting a model of a frame of a pair of smart glasses from a plurality of models of frames for placement on the face image of the person, the pair of smart glasses including a display embedded in, or overlaid on a lens of the pair of smart glasses; sizing the selected model of the frame of the pair of smart glasses for placement on the face image of person based on the lens measurements; and determining a gazability measure of the display by adjusting a gaze of a pupil of the person to each corner of the display and determining how much of a light transmitted through the display is entering the pupil. ### Claim 9 (depends on 8) The method of claim 8, wherein the received lens measurements include one or more of a left pupillary distance, a right pupillary distance, a pupil height, a pantoscopic angle, and a vertex distance. ### Claim 10 (depends on 8) The method of claim 8, wherein the received face image of the person is a frontal face image of the person. ### Claim 11 (depends on 8) The method of claim 8, wherein the selected model of the frame is selected from a frames library, the frames library including the plurality of the models of the frames. ### Claim 12 (depends on 8) The method of claim 8, wherein sizing the selected model of the frame of the pair of smart glasses for placement on the person includes placing the selected model of the frame in 3-D space on or aligned with a 3-D model of a head of the person. ### Claim 13 (depends on 12) The method of claim 12, wherein placing the selected model of the frame in 3-D space on or aligned with the 3-D model of the head of the person includes determining a location of a pupil of the person by combining information available in the lens measurements including a right pupillary distance (RPD), a pupil height, and a vertex distance. ### Claim 14 (depends on 8) The method of claim 8, wherein determining gazability of the display includes conducting mathematical operations based on the lens measurements to position a pupil of the person within an eye box, and then adjusting a gaze of the pupil to each corner of the display and determining how much of a display light is entering the pupil. ### Claim 15 (depends on 14) The method of claim 14, wherein receiving the face image of a person includes receiving side-view face images of the person, and aligning and placing the sized model of the frame of the pair of smart glasses on the side-view face images of the person. ### Claim 16 A non-transitory computer-readable medium storing executable instructions that when executed by at least one processor are configured to cause the at least one processor to: receive a face image of a person; receive lens measurements of the person; select a model of a frame of a pair of smart glasses from a plurality of models of frames for placement on the face image of the person, the pair of smart glasses including a display embedded in, or overlaid on a lens of the pair of smart glasses; size the selected model of the frame of the pair of smart glasses for placement on the face image of person based on the lens measurements; and determine a gazability measure of the display by adjusting a gaze of a pupil of the person to each corner of the display and determining how much of a light transmitted through the display is entering the pupil. ### Claim 17 (depends on 16) The non-transitory computer-readable medium of claim 16, wherein the received lens measurements include one or more of a left pupillary distance, a tight pupillary distance, a pupil height, a pantoscopic angle, and a vertex distance. ### Claim 18 (depends on 16) The non-transitory computer-readable medium of claim 16, wherein the received face image of the person is a frontal face image of the person. ### Claim 19 (depends on 16) The non-transitory computer-readable medium of claim 16, wherein the selected model of the frame is selected from a frames library, the frames library including the plurality of the models of the frames. ### Claim 20 (depends on 16) The non-transitory computer-readable medium of claim 16, wherein sizing the selected model of the frame of the pair of smart glasses for placement on the face image of the person includes placing the selected model of the frame in 3-D space on or aligned with a 3-D model of a head of the person. ### Claim 21 (depends on 20) The non-transitory computer-readable medium of claim 20, wherein placing the selected model of the frame in 3-D space on or aligned with the 3-D model of the head of the person includes determining a location of a pupil of the person by combining information available in the lens measurements including a right pupillary distance (RPD), a pupil height, and a vertex distance. ### Claim 22 (depends on 16) The non-transitory computer-readable medium of claim 16, wherein receiving the face image of a person includes receiving side-view face images of the person, and aligning and placing the sized model of the frame of the pair of smart glasses on the side-view face images of the person.