# US 2023/0053497 A1 · Systems and methods for performing eye-tracking > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2023/0053497 A1 - Document ID: US20230053497A1 - Status: Published application - Kind: A1 - Application number: 17/724,302 - Filed: 2022-04-19 - Published: 2023-02-23 - Earliest claimed: 2021-08-18 - Assignees: Meta Platforms Technologies, LLC - Applicants: Meta Platforms Technologies, LLC - Inventors: Syed Shakib Sarwar; Barbara De Salvo; Xinqiao Liu - Web page: https://patentgrep.com/patent/US20230053497A1 ## CPC - A61B 3/113: for determining or recording eye movement - G06T 7/11: Region-based segmentation - G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI] - G06V 10/94: Hardware or software architectures specially adapted for image or video understanding - G06V 10/993: {Evaluation of the quality of the acquired pattern} - G06V 40/19: Sensors therefor - G06T 2207/10024: Color image - G06T 2207/10048: Infrared image - G06T 2207/20132: Image cropping - G06T 2207/30041: Eye; Retina; Ophthalmic - G06T 2207/30201: Face - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract The disclosed computer-implemented method may include (i) conditionally operating, at a first frequency, a first stage of an eye-tracking system processing pipeline that detects a region of interest and (ii) operating, at a second frequency that is substantially greater than the first frequency, a second stage of the eye-tracking system processing pipeline that predicts a gaze orientation based at least in part on the detected region of interest. Various other methods, systems, and computer-readable media are also disclosed. ## Claims ### Claim 1 An eye-tracking headset comprising: a physical processor; and at least one physical memory storing executable instructions that, when executed by the physical processor, cause the physical processor to: conditionally operate, at a first frequency, a first stage of an eye-tracking system processing pipeline that detects a region of interest; and operate, at a second frequency that is substantially greater than the first frequency, a second stage of the eye-tracking system processing pipeline that predicts a gaze orientation based at least in part on the detected region of interest; wherein: the first stage updates information identifying a crop location corresponding to a human eye; and the second stage performs, at the second frequency that is substantially greater than the first frequency, a cropping procedure using the crop location previously updated from the first stage that is operating at the first frequency. ### Claim 2 (depends on 1) The eye-tracking headset of claim 1, wherein the executable instructions further cause the physical processor to detect an indication of a corner case that threatens to impede performance of the eye-tracking system processing pipeline. ### Claim 3 (depends on 2) The eye-tracking headset of claim 2, wherein the indication of the corner case comprises: movement; or a quality measurement of output of the second stage falling below a threshold. ### Claim 4 (depends on 3) The eye-tracking headset of claim 3, wherein the indication of the corner case comprises movement. ### Claim 5 (depends on 4) The eye-tracking headset of claim 4, further comprising an inertial measurement unit that is configured to detect movement. ### Claim 6 (depends on 5) The eye-tracking headset of claim 5, wherein the inertial measurement unit is configured to detect a quantity of movement that satisfies a predetermined threshold. ### Claim 7 (depends on 4) The eye-tracking headset of claim 4, wherein the executable instructions further cause the physical processor to deviate, in response to detecting movement, from the first frequency by activating the first stage for an extra frame such that an accuracy of detecting the region of interest is improved. ### Claim 8 (depends on 4) The eye-tracking headset of claim 4, wherein the executable instructions further cause the physical processor to deviate, in response to detecting movement, from a subsampling procedure that subsamples the region of interest to a subsampling procedure that subsamples an entire frame that includes the region of interest. ### Claim 9 (depends on 3) The eye-tracking headset of claim 3, wherein the indication of the corner case comprises the quality measurement of output of the second stage falling below the threshold. ### Claim 10 (depends on 9) The eye-tracking headset of claim 9, wherein the executable instructions further cause the physical processor to deviate, in response to detecting the quality measurement of output of the second stage falling below the threshold, from a subsampling procedure that subsamples the region of interest to a subsampling procedure that subsamples an entire frame that includes the region of interest. ### Claim 11 (depends on 3) The eye-tracking headset of claim 3, wherein the executable instructions further cause the physical processor to execute an inclusive OR operation that, in response to detecting either movement or detecting the quality measurement of output of the second stage falling below the threshold, triggers deviating from a subsampling procedure that subsamples the region of interest to a subsampling procedure that subsamples an entire frame that includes the region of interest. ### Claim 12 (depends on 3) The eye-tracking headset of claim 3, wherein the executable instructions further cause the physical processor to detect the indication of the corner case that is detected by a sensor other than a camera sensor. ### Claim 13 (depends on 1) The eye-tracking headset of claim 1, wherein the eye-tracking system processing pipeline is configured to operate as part of a virtual/augmented reality headset. ### Claim 14 (depends on 1) The eye-tracking headset of claim 1, wherein the executable instructions further cause the physical processor to conditionally operate the first stage of the eye-tracking system processing pipeline that detects the region of interest at the first frequency such that power consumption is achieved that is substantially lower than in comparison to unconditionally operating the first stage. ### Claim 15 (depends on 14) The eye-tracking headset of claim 14, wherein the executable instructions further cause the physical processor to conditionally operate the first stage of the eye-tracking system processing pipeline that detects the region of interest at the first frequency such that power consumption is reduced from hundreds of milliwatts to single digit milliwatts in comparison to operating the first stage at the second frequency. ### Claim 16 (depends on 1) The eye-tracking headset of claim 1, wherein the executable instructions further cause the physical processor to operate the first stage of the eye-tracking system processing pipeline that detects the region of interest at the first frequency such that latency is improved in comparison to operating the first stage at the second frequency. ### Claim 17 (depends on 1) The eye-tracking headset of claim 1, wherein the executable instructions further cause the physical processor to select statically the first frequency based on heuristics gathered from data analysis. ### Claim 18 (depends on 1) The eye-tracking headset of claim 1, wherein the executable instructions further cause the physical processor to execute a machine learning algorithm to predict the gaze orientation. ### Claim 19 A method comprising: conditionally operating, at a first frequency, a first stage of an eye-tracking system processing pipeline that detects a region of interest; and operating, at a second frequency that is substantially greater than the first frequency, a second stage of the eye-tracking system processing pipeline that predicts a gaze orientation based at least in part on the detected region of interest; wherein: the first stage updates information identifying a crop location corresponding to a human eye; and the second stage performs, at the second frequency that is substantially greater than the first frequency, a cropping procedure using the crop location previously updated from the first stage that is operating at the first frequency. ### Claim 20 A non-transitory computer-readable medium comprising one or more computer-executable instructions that, when executed by at least one processor of a computing device, cause the computing device to: conditionally operate, at a first frequency, a first stage of an eye-tracking system processing pipeline that detects a region of interest; and operate, at a second frequency that is substantially greater than the first frequency, a second stage of the eye-tracking system processing pipeline that predicts a gaze orientation based at least in part on the detected region of interest; wherein: the first stage updates information identifying a crop location corresponding to a human eye; and the second stage performs, at the second frequency that is substantially greater than the first frequency, a cropping procedure using the crop location previously updated from the first stage that is operating at the first frequency.