# US 12,468,387 B2 · Operating an eye tracker camera according to eye position > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,468,387 B2 - Document ID: US12468387B2 - Status: Granted - Kind: B2 - Application number: 18/473,230 - Filed: 2023-09-23 - Published: 2024-01-11 - Granted: 2025-11-11 - Earliest claimed: 2020-11-14 - Assignees: Facense Ltd. - Applicants: Facense Ltd. - Inventors: Gil Thieberger; Ari M Frank - Web page: https://patentgrep.com/patent/US12468387B2 ## CPC - G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)} - A61B 5/0205: Simultaneously evaluating both cardiovascular conditions and different types of body conditions, e.g. heart and respiratory condition - A61B 5/02427: {Details of sensor} - A61B 5/02438: {with portable devices, e.g. worn by the patient} - A61B 5/1103: {Detecting muscular movement of the eye, e.g. eyelid movement} - A61B 5/1128: {using image analysis (A61B5/1127 takes precedence)} - A61B 5/14546: {for measuring analytes not otherwise provided for, e.g. ions, cytochromes} - A61B 5/1455: using optical sensors, e.g. spectral photometrical oximeters - A61B 5/163: {by tracking eye movement, gaze, or pupil change} - A61B 5/6803: {Head-worn items, e.g. helmets, masks, headphones or goggles} - G06V 10/141: Control of illumination - G06V 10/94: Hardware or software architectures specially adapted for image or video understanding - G06V 40/166: {using acquisition arrangements} - G06V 40/174: {Facial expression recognition} - G06V 40/19: Sensors therefor - H04N 23/611: where the recognised objects include parts of the human body - H04N 23/651: {for reducing power consumption by affecting camera operations, e.g. sleep mode, hibernation mode or power off of selective parts of the camera} - H04N 23/667: Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes - H04N 23/951: by using two or more images to influence resolution, frame rate or aspect ratio - A61B 5/02416: {using photoplethysmograph signals, e.g. generated by infrared radiation (A61B5/14552 takes precedence)} - A61B 5/7221: {Determining signal validity, reliability or quality (preventing, reducing or removing noise induced by motion artefacts A61B5/7207; noise originating from a therapeutic or surgical apparatus A61B5/7217)} - H04N 25/46: by combining or binning pixels - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Systems, methods, and computer programs for efficient use of head-mounted cameras in eye tracking. In one embodiment, a system includes a head-mounted device that takes measurements indicative of positions of an eye of a user, and a head-mounted camera that captures images of the eye. A first range of eye positions trackable from the images is narrower than a second range of eye positions trackable from the measurements. A computer calculates the eye positions based on the measurements. When the eye positions fall within the first range, the computer reads images at a first bitrate from the head-mounted camera. And when the eye positions fall outside the first range, the computer refrains from reading images from the head-mounted camera, or reads the images from the head-mounted camera at a second bitrate that is less than half the first bitrate. ## Claims ### Claim 1 An eye tracking system comprising: a head-mounted device configured to take measurements indicative of positions of an eye of a user (eye positions); a head-mounted camera configured to capture images of the eye; whereby a first range of eye positions trackable from the images is narrower than a second range of eye positions trackable from the measurements; and a computer configured to: calculate eye positions based on the measurements; read the images from the head-mounted camera at a first bitrate when the eye positions fall within the first range; refrain from reading images from the head-mounted camera, or read the images from the head-mounted camera at a second bitrate that is less than half the first bitrate, when the eye positions fall outside the first range; and wherein the eye positions that fall outside the first range span at least 20° of the eve's field of view, and the computer is further configured to calculate the eye positions based on the measurements at a rate that is higher than a rate at which the images are captured by the head-mounted camera. ### Claim 2 (depends on 1) The eye tracking system of claim 1, wherein the computer is further configured to calculate the eye positions based on the measurements at a rate that is at least ten times higher than a rate at which the images are captured by the head-mounted camera, and to command the head-mounted camera to operate in a low-power mode for a longer percent of time while the eye positions fall outside the first range compared to a percent of time it operates in low-power mode while the eye positions fall within the first range. ### Claim 3 (depends on 1) The eye tracking system of claim 1, further comprising a second head-mounted camera configured to capture a second set of images of the eye from a different position than the head-mounted camera; and wherein the computer is further configured to read the second set of images from the second head-mounted camera at a higher bitrate when the eye positions fall outside the first range compared to when the eye positions fall within the first range. ### Claim 4 (depends on 3) The eye tracking system of claim 3, wherein the computer is further configured to command the head-mounted camera to operate in a low-power mode for a longer percent of time while the eye positions fall outside the first range compared to a percent of time it operates in the low-power mode while the eye positions fall within the first range; and wherein the computer is further configured to command the second head-mounted camera to operate in a low-power mode for a longer percent of time while the eye positions fall within the first range compared to a percent of time it operates in the low-power mode while the eye positions fall outside the first range. ### Claim 5 (depends on 3) The eye tracking system of claim 3, wherein the head-mounted camera and the second head-mounted camera are connected to the computer over a bus that does not have sufficient bandwidth to transfer images from both cameras at a maximum bitrate at which the computer is capable of reading from each camera; and wherein the head-mounted camera and the second head-mounted camera are positioned at least 2 cm apart horizontally. ### Claim 6 (depends on 1) The eye tracking system of claim 1, wherein the eye is the user's right eye, the head-mounted camera is located to the right of the lateral canthus of the eye, and eye positions in vicinity of the eye's medial canthus are outside the first range. ### Claim 7 (depends on 1) The eye tracking system of claim 1, wherein the head-mounted camera is capable of tilting and/or shifting its sensor relative to its optics according to the Scheimpflug principle; and the computer is further configured to command the head-mounted camera to perform the tilting and/or shifting according to the eye positions. ### Claim 8 (depends on 1) The eye tracking system of claim 1, wherein the head-mounted device comprises a photosensor-oculography device (PSOG), and the measurements are of reflections of light emitted by the PSOG towards the eye. ### Claim 9 (depends on 8) The eye tracking system of claim 8, wherein the computer is further configured to generate feature values based on data comprising measurements of the reflections and the images, and to utilize a model to calculate the eye positions based on the feature values. ### Claim 10 (depends on 8) The eye tracking system of claim 8, wherein the computer is further configured to (i) extract a set of eyelid positions based on analyzing the images, and (ii) provide labels that are based on the set of eyelid positions, and corresponding feature values that are based on the emitted light and measurements of the reflections, to train a model for calculating eyelid position based on additional measurements of reflections. ### Claim 11 (depends on 1) The eye tracking system of claim 1, wherein the head-mounted device comprises an electrooculography device, and the measurements comprise values of electrical potentials between electrodes placed close to the eye. ### Claim 12 (depends on 1) The eye tracking system of claim 1, wherein the head-mounted device comprises an electromyography device, and the measurements comprise values of electrical potentials generated by muscle cells. ### Claim 13 (depends on 1) The eye tracking system of claim 1, wherein the head-mounted device comprises an optical flow sensor, the measurements comprise values of optical flow and/or visual motion, and the eye positions are calculated based on an optical flow algorithm. ### Claim 14 (depends on 1) The eye tracking system of claim 1, wherein the head-mounted device comprises range sensors, and the measurements comprise values indicative of ranges between the range sensors and the eye. ### Claim 15 A method comprising: taking, with a head-mounted device, measurements indicative of positions of an eye of a user (eye positions); capturing images of the eye by a head-mounted camera; whereby a first range of eye positions trackable from the images is narrower than a second range of eye positions trackable from the measurements; calculating eye positions based on the measurements; reading the images from the head-mounted camera at a first bitrate when the eye positions fall within the first range; refraining from reading the images from the head-mounted camera, or reading the images from the head-mounted camera at a second bitrate that is less than half the first bitrate, when the eye positions fall outside the first range; capturing, by a second head-mounted camera, a second set of images of the eye from a different position than the head-mounted camera; and reading the second set of images from the second head-mounted camera at a higher bitrate when the eye positions fall outside the first range compared to when the eye positions fall within the first range. ### Claim 16 (depends on 15) The method of claim 15, further comprising commanding the head-mounted camera to operate in a low-power mode for a longer percent of time while the eye positions fall outside the first range compared to a percent of time it operates in low-power mode while the eye positions fall within the first range. ### Claim 17 A non-transitory computer readable medium storing one or more computer programs configured to cause a processor-based system to execute steps comprising: taking, with a head-mounted device, measurements indicative of positions of an eye of a user (eye positions); capturing images of the eye by a head-mounted camera; whereby a first range of eye positions trackable from the images is narrower than a second range of eye positions trackable from the measurements; calculating eye positions based on the measurements; reading the images from the head-mounted camera at a first bitrate when the eye positions fall within the first range; refraining from reading the images from the head-mounted camera, or reading the images from the head-mounted camera at a second bitrate that is less than half the first bitrate, when the eye positions fall outside the first range; capturing, by a second head-mounted camera, a second set of images of the eye from a different position than the head-mounted camera; and reading the second set of images from the second head-mounted camera at a higher bitrate when the eye positions fall outside the first range compared to when the eye positions fall within the first range.