# US 2022/0035449 A1 · Gaze tracking system and method > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2022/0035449 A1 - Document ID: US20220035449A1 - Status: Published application - Kind: A1 - Application number: 17/380,182 - Filed: 2021-07-20 - Published: 2022-02-03 - Earliest claimed: 2020-07-28 - Assignees: Sony Interactive Entertainment Inc. - Applicants: Sony Interactive Entertainment Inc. - Inventors: Philip Cockram - Web page: https://patentgrep.com/patent/US20220035449A1 ## CPC - G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)} - G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking} - G02B 27/0101: {characterised by optical features (G02B27/0172 takes precedence)} - G02B 27/017: {Head mounted} - G02B 27/0179: {Display position adjusting means not related to the information to be displayed} - G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)} - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G06V 40/19: Sensors therefor - G02B 2027/0138: {comprising image capture systems, e.g. camera} - G02B 2027/0187: {slaved to motion of at least a part of the body of the user, e.g. head, eye} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A gaze tracking system comprising a pupil detection unit operable to detect the location of a pupil in one or more images captured of one or both a user's eyes, a confidence value determination unit operable to determine a confidence value in dependence upon the identified pupil position, the confidence value indicating an expected reliability of the detection of the pupil location, and a processing unit operable to generate, in dependence upon the determined confidence value, one or more outputs to cause the user to modify their pupil location to a location with a higher determined confidence value. ## Claims ### Claim 1 A gaze tracking system comprising: a pupil detection unit operable to detect the location of a pupil in one or more images captured of one or both a user's eyes; a confidence value determination unit operable to determine a confidence value in dependence upon the identified pupil position, the confidence value indicating an expected reliability of the detection of the pupil location; and a processing unit operable to generate, in dependence upon the determined confidence value, one or more outputs to cause the user to modify their pupil location to a location with a higher determined confidence value. ### Claim 2 (depends on 1) The system of claim 1, wherein the processing unit is operable to generate an audio output. ### Claim 3 (depends on 1) The system of claim 1, wherein the processing unit is operable to generate a visual output. ### Claim 4 (depends on 3) The system of claim 3, wherein the visual output comprises a user interface element. ### Claim 5 (depends on 3) The system of claim 3, wherein the visual output comprises an optical effect that is applied to one or more peripheral regions of an image for display to the user. ### Claim 6 (depends on 3) The system of claim 3, wherein the visual output comprises a virtual object within content being viewed by the user. ### Claim 7 (depends on 1) The system of claim 1, comprising a pupil motion prediction unit operable to predict future motion of one or both of the user's pupils and to determine confidence values corresponding to the positions corresponding to the predicted motion, wherein the processing unit is operable to generate the one or more outputs in dependence upon the predicted motion in addition to, or instead of, the detected pupil location. ### Claim 8 (depends on 7) The system of claim 7, wherein the predictions are based upon one or more of a current pupil position, current pupil velocity, past pupil motion, and information about content being viewed by the user. ### Claim 9 (depends on 1) The system of claim 1, wherein one or more aspects of the one or more generated outputs is varied proportionally to the determined confidence value. ### Claim 10 (depends on 1) The system of claim 1, wherein the expected reliability is calculated in dependence upon one or more of pupil distance from a camera used to capture images of the eye, the relative angle of the pupil to the camera, and the amount and/or angle of light incident on the eye. ### Claim 11 (depends on 1) The system of claim 1, comprising a head-mountable display unit operable to display one or more images to the user and to provide the one or more outputs to the user. ### Claim 12 (depends on 1) The system of claim 1, comprising one or more cameras operable to capture images of one or both of the user's eyes. ### Claim 13 A gaze tracking method comprising: detecting the location of a pupil in one or more images captured of one or both a user's eyes; determining a confidence value in dependence upon the identified pupil position, the confidence value indicating an expected reliability of the detection of the pupil location; and generating, in dependence upon the determined confidence value, one or more outputs to cause the user to modify their pupil location to a location with a higher determined confidence value. ### Claim 14 A non-transitory machine-readable storage medium which stores computer software which, when executed by a computer, causes the computer to perform a gaze tracking method comprising: detecting the location of a pupil in one or more images captured of one or both a user's eyes; determining a confidence value in dependence upon the identified pupil position, the confidence value indicating an expected reliability of the detection of the pupil location; and generating, in dependence upon the determined confidence value, one or more outputs to cause the user to modify their pupil location to a location with a higher determined confidence value.