# US 11,442,542 B2 · Systems and methods for eye tracking > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,442,542 B2 - Document ID: US11442542B2 - Status: Granted - Kind: B2 - Application number: 17/365,838 - Filed: 2021-07-01 - Published: 2021-10-21 - Granted: 2022-09-13 - Earliest claimed: 2020-01-27 - Assignees: Valve Corporation - Applicants: Valve Corporation - Inventors: Joshua Hudman; Alan Yates; Gordon Wayne Stoll; Evan Fletcher - Web page: https://patentgrep.com/patent/US11442542B2 ## CPC - G02B 27/0172: {characterised by optical features} - G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)} - G01J 1/0429: {using polarisation elements} - G01J 1/42: using electric radiation detectors (optical or mechanical part G01J1/04; by comparison with a reference light or electric value G01J1/10) - G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking} - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/012: {Head tracking input arrangements} - G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)} - G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] - G06N 3/045: Combinations of networks - G06N 3/047: Probabilistic or stochastic networks - G06N 3/08: Learning methods - G06N 3/09: Supervised learning - G02B 5/3025: {Polarisers, i.e. arrangements capable of producing a definite output polarisation state from an unpolarised input state (G02B5/3008, G02B5/3016 take precedence)} - G02B 2027/0187: {slaved to motion of at least a part of the body of the user, e.g. head, eye} - G06N 3/044: Recurrent networks, e.g. Hopfield networks - G06N 20/00: Machine learning - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G01J: MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY (light sources F21, H01J, H01K, H05B; investigating properties of materials by optical means G01N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract The present disclosure is directed to systems and methods of eye tracking for use in various applications, such as virtual reality or augmented reality applications that include head-mounted display devices. An eye tracking subsystem may be provided that includes a plurality of assemblies that each include a light source, such as an LED, a light detector, such as a silicon photodiode, and a polarizer that is configured to prevent light reflected via specular reflection from reaching the light detectors so that only scattered light is detected by the light detectors. Machine learning or other techniques may be used to track or otherwise determine a user's gaze direction, which may be used by one or more components of an HMD device to improve its functionality in various ways. ## Claims ### Claim 1 An eye tracking system, comprising: a plurality of eye tracking assemblies, each of the plurality of eye tracking assemblies includes: a light source; a light detector; and a polarizer positioned proximate to at least one of the light source and the light detector, the polarizer configured to prevent light reflected via specular reflection from being received by the light detector while allowing light reflected via diffuse reflection to be received by the light detector, wherein light reflected via specular reflection comprises light that is reflected from a surface at an angle equal to the angle of incidence, and light reflected via diffuse reflection comprises light that is scattered from a surface at many angles. ### Claim 2 (depends on 1) The eye tracking system of claim 1 wherein each of the light sources is directed toward an expected location of a pupil of a user. ### Claim 3 (depends on 1) The eye tracking system of claim 1 wherein the light sources comprise light emitting diodes that emit light having a wavelength that is between 780 nm and 1000 nm. ### Claim 4 (depends on 1) The eye tracking system of claim 1 wherein the light detectors comprise photodiodes. ### Claim 5 (depends on 1) The eye tracking system of claim 1 wherein the eye tracking system comprises four eye tracking assemblies positioned to determine a gaze direction of a left eye of a user and four eye tracking assemblies positioned to determine a gaze direction of a right eye of the user. ### Claim 6 (depends on 1) The eye tracking system of claim 1 wherein the polarizer comprises two cross linear polarizers. ### Claim 7 (depends on 1) The eye tracking system of claim 1 wherein, for each of the plurality of eye tracking assemblies, the polarizer comprises a first polarizer positioned in a light emitting path of the light source and a second polarizer positioned in a light detecting path of the light detector. ### Claim 8 (depends on 1) The eye tracking system of claim 1 wherein the polarizer comprises at least one of a circular polarizer or a linear polarizer. ### Claim 9 (depends on 1) The eye tracking system of claim 1 wherein, for each of the plurality of eye tracking assemblies, the light source is positioned away from an optical axis of an eye of a user to provide dark field illumination of the pupil of the eye. ### Claim 10 An eye tracking system, comprising: a plurality of eye tracking assemblies, each of the plurality of eye tracking assemblies includes: a light source; a light detector; and a polarizer positioned proximate to at least one of the light source and the light detector, the polarizer configured to prevent light reflected via specular reflection from being received by the light detector while allowing light reflected via diffuse reflection to be received by the light detector, wherein light reflected via specular reflection comprises light that is reflected from a surface at an angle equal to the angle of incidence, and light reflected via diffuse reflection comprises light that is scattered from a surface at many angles; control circuitry that, in operation, causes the eye tracking system to: selectively cause the light sources of the plurality of eye tracking assemblies to emit light; receive light detection information captured by the light detectors of the plurality of eye tracking assemblies; provide the received light detection information as input to a prediction model; receive, from the prediction model in response to providing the light detection information, a determined gaze direction for an eye of a user; and store the determined gaze direction in a nontransitory memory. ### Claim 11 (depends on 10) The eye tracking system of claim 10 wherein the light detection information comprises a signature radiation pattern of each of the light sources after the light from the light source has reflected, scattered, or been absorbed of off the face or eye of a user. ### Claim 12 (depends on 10) The eye tracking system of claim 10 wherein each of the light sources is directed toward an expected location of a pupil of a user. ### Claim 13 (depends on 10) The eye tracking system of claim 10 wherein the light sources comprise light emitting diodes that emit light having a wavelength that is between 780 nm and 1000 nm. ### Claim 14 (depends on 10) The eye tracking system of claim 10 wherein the light detectors comprise photodiodes. ### Claim 15 (depends on 10) The eye tracking system of claim 10 wherein the eye tracking system comprises four eye tracking assemblies positioned to determine the gaze direction of a left eye of a user and four eye tracking assemblies positioned to determine the gaze direction of a right eye of the user. ### Claim 16 (depends on 10) The eye tracking system of claim 10 wherein the polarizer comprises two cross linear polarizers. ### Claim 17 (depends on 10) The eye tracking system of claim 10 wherein, for each of the plurality of eye tracking assemblies, the polarizer comprises a first polarizer positioned in a light emitting path of the light source and a second polarizer positioned in a light detecting path of the light detector. ### Claim 18 (depends on 10) The eye tracking system of claim 10 wherein the polarizer comprises at least one of a circular polarizer or a linear polarizer. ### Claim 19 (depends on 10) The eye tracking system of claim 10 wherein, for each of the plurality of eye tracking assemblies, the light source is positioned away from an optical axis of an eye of a user to provide dark field illumination of the pupil of the eye. ### Claim 20 (depends on 10) The eye tracking system of claim 10 wherein the prediction model comprises a machine learning model. ### Claim 21 (depends on 10) The eye tracking system of claim 10, further comprising at least one display, wherein, in operation, the control circuitry: causes the at least one display to present a user interface element; selectively causes the light sources of the plurality of eye tracking assemblies to emit light; receives light detection information captured by the light detectors of the plurality of eye tracking assemblies; and updates the prediction model based at least in part on the received light detection information and known or inferred gaze direction information associated with the received light detection information. ### Claim 22 (depends on 21) The eye tracking system of claim 21 wherein the user interface element includes a static user interface element or a moving user interface element. ### Claim 23 (depends on 21) The eye tracking system of claim 21 further comprising at least one display, wherein, in operation, the control circuitry: dynamically modifies a rendering output of the at least one display based at least in part on the determined gaze direction. ### Claim 24 (depends on 10) The eye tracking system of claim 10, further comprising an interpupillary distance (IPD) adjustment component, and, in operation, the control circuitry: causes the IPD adjustment component to align at least one component of the eye tracking system for a user based at least in part on the determined gaze direction. ### Claim 25 A method of operating an eye tracking system that comprises a plurality of eye tracking assemblies that each include a light source, a light detector, and a polarizer, the method comprising: selectively causing the plurality of light sources of the plurality of eye tracking assemblies to emit light; preventing, by each of the respective polarizers of the plurality of eye tracking assemblies, light reflected via specular reflection from being received by the plurality of light detectors while allowing light reflected via diffuse reflection to be received by the plurality of light detectors, wherein light reflected via specular reflection comprises light that is reflected from a surface at an angle equal to the angle of incidence, and light reflected via diffuse reflection comprises light that is scattered from a surface at many angles; receiving light detection information captured by the plurality of light detectors, the light detection information comprising the light reflected by diffuse reflection; providing the received light detection information as input to a trained machine learning model; receiving, from the trained machine learning model in response to providing the light detection information, a determined gaze direction for an eye of a user; and storing the determined gaze direction in a nontransitory memory. ### Claim 26 (depends on 25) The method of claim 25 wherein the light detection information comprises a signature radiation pattern of each of the light sources after the light from the light source has reflected, scattered, or been absorbed of off the face or eye of the user. ### Claim 27 (depends on 25) The method of claim 25 wherein providing the received light detection information as input to the trained machine learning model comprises providing the received light detection information as input to a mixture density network (MDN) model. ### Claim 28 (depends on 25) The method of claim 25 wherein providing the received light detection information as input to the trained machine learning model comprises providing the received light detection information as input to a recurrent neural network (RNN) model. ### Claim 29 (depends on 25) The method of claim 25 wherein providing the received light detection information as input to the trained machine learning model comprises providing the received light detection information to a machine learning model that utilizes past input information or eye motion information to determine the gaze direction. ### Claim 30 (depends on 25) The method of claim 25, further comprising training the trained machine learning model during field operation of a plurality of eye tracking systems. ### Claim 31 (depends on 25) The method of claim 25 wherein the eye tracking system comprises at least one display, and the method comprises: dynamically modifying an output of the at least one display based at least in part on the determined gaze direction. ### Claim 32 (depends on 25) The method of claim 25, further comprising: mechanically aligning at least one component of an HMD system for the user based at least in part on the determined gaze direction.