# US 2021/0344896 A1 · Dynamic Illumination for Eye-Tracking > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0344896 A1 - Document ID: US20210344896A1 - Status: Published application - Kind: A1 - Application number: 17/271,522 - Filed: 2019-08-28 - Published: 2021-11-04 - Earliest claimed: 2018-08-29 - Assignees: Intuitive Surgical Operations, Inc. - Applicants: Intuitive Surgical Operations, Inc. - Inventors: Heath Feather; Randal P. Goldberg; Alexandre V. Matveev - Web page: https://patentgrep.com/patent/US20210344896A1 ## CPC - G06V 40/18: Eye characteristics, e.g. of the iris - H04N 13/383: for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes - A61B 3/113: for determining or recording eye movement - G02B 1/11: Anti-reflection coatings - G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking} - G02B 27/0172: {characterised by optical features} - G02B 30/35: using reflective optical elements in the optical path between the images and the observer - H04N 13/344: with head-mounted left-right displays - H04N 23/56: provided with illuminating means - G02B 2027/0134: {of stereoscopic type} - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS ## Abstract An apparatus includes a controller coupled to a stereoscopic image viewer. The stereoscopic image viewer includes a viewing lens, an image capture unit, and a plurality of light sources. The image capture unit is configured to capture image frames. The controller receives the captured image frames. The controller is configured to switch light output from one of the plurality of light sources to another one of the plurality of light sources if a corneal reflection in a captured image frame intersects a fixed reflection in the captured image frame. ## Claims ### Claim 1 An apparatus comprising: a viewing lens; a display device configured to output a visible light image for viewing by a user through the viewing lens; a plurality of light sources configured to output light used in tracking a gaze of the user while the user is viewing the visible light image through the viewing lens; an image capture; a reflective structure configured to reflect the visible light image output by the display device through the viewing lens, and pass a corneal reflection from the user to the image capture unit; and a controller configured to: generate the corneal reflection using a first one of the plurality of light sources; and switch to a second one of the plurality of light sources to generate the corneal reflection if the corneal reflection generated by the first one of the plurality of light sources intersects a fixed reflection that is from the viewing lens and that is captured by the image capture unit. ### Claim 2 (depends on 1) The apparatus of claim 1, wherein the viewing lens comprises: an outer surface; and an anti-reflective coating for the light from the plurality of light sources, the anti-reflective coating being on the outer surface of the viewing lens. ### Claim 3 (depends on 1) The apparatus of claim 1, wherein the plurality of light sources comprises a plurality of non-visible light sources. ### Claim 4 (canceled) ### Claim 5 (depends on 1) The apparatus of claim 1: wherein the image capture unit comprises a lens assembly; and wherein the first one of the plurality of light sources is displaced in a first direction from the lens assembly of the image capture unit; and the second one of the plurality of light sources is displaced in a second direction from the lens assembly of the image capture unit, where the first direction is opposite to the second direction. ### Claim 6 (canceled) ### Claim 7 An apparatus comprising: a first lens and a second lens; a first display device configured to output a first visible light image for viewing by a user through the first lens; a second display device configured to output a second visible light image for viewing by the user through the second lens; an image capture unit; a plurality of light sources configured to illuminate a left eye and a right eye of the user with light after the light passes through the first and second lenses, the light from the plurality of light sources used in tracking a gaze of the user while the user is viewing the first and second visible light images through the first and second lenses; a first reflective structure configured to reflect the first visible light image output by the first display device through the first lens, and pass a first corneal reflection from the right eye to the image capture unit; a second reflective structure configured to reflect the second visible light image output by the second display device through the second lens, and pass a second corneal reflection from the left eye to the image capture unit; and a controller configured to: generate at least one of the first corneal reflection or the second corneal reflection using a first one of the plurality of light sources; and stop light output from the first one of the plurality of light sources if the light output from the first one of the plurality of light sources results in either the first corneal reflection or the second corneal reflection intersecting a fixed reflection that is from either the first lens or the second lens and that is captured by the image capture unit. ### Claim 8 (depends on 7) The apparatus of claim 7, further comprising: a first movable viewing assembly comprising the first lens and the first reflective structure, the first reflective structure passing a first portion of the light from the plurality of light sources; and a second movable viewing assembly comprising the second lens and the second reflective structure, the second reflective structure passing a second portion of the light from the plurality of light sources. ### Claim 9 (depends on 7) The apparatus of claim 7, wherein the first reflective structure comprises a first dichroic surface, and wherein the second reflective structure comprises a second dichroic surface. ### Claim 10 (canceled) ### Claim 11 (depends on 8) The apparatus of claim 8: the first lens being fixedly coupled to the first reflective structure in the first movable viewing assembly, wherein the first lens and the first reflective structure move as a unit as the first movable viewing assembly moves; and the second lens being fixedly coupled to second reflective structure in the second movable viewing assembly, wherein the second lens and the second reflective structure move as a unit as the second movable viewing assembly moves. ### Claim 12 (depends on 11) The apparatus of claim 11, wherein: the first movable viewing assembly further comprises the first display device mounted in the first movable viewing assembly to provide the first visible light image to the first reflective structure, the first display device being fixedly coupled to the first reflective structure in the first movable viewing assembly, wherein the first lens, the first reflective structure, and the first display device move as a unit as the first movable viewing assembly moves; and the second movable viewing assembly further comprises the second display device mounted in the second movable viewing assembly to provide the second visible light image to the second reflective structure, the second display device being fixedly coupled to the second reflective structure in the second movable viewing assembly, wherein the second lens, the second reflective structure, and the second display device move as a unit as the second movable viewing assembly moves. ### Claim 13 (depends on 7) The apparatus of claim 7, wherein each of the first lens and the second lens comprises a plurality of lenses combined to together to form a single lens structure. ### Claim 14 (depends on 13) The apparatus of claim 13, wherein the single lens structure comprises an outer surface, and an anti-reflective coating for the light from the plurality of light sources on the outer surface. ### Claim 15 (depends on 7) The apparatus of claim 7, the controller further being configured to cause a different one of the of the plurality of light sources to output the light if the light output from the first one of the plurality of light sources is stopped. ### Claim 16 (depends on 7) The apparatus of claim 7, wherein the plurality of light sources comprises a first set of light sources and a second set of light sources, the first set of light source being configured to illuminate the left eye with the light after the light passes through the first lens, and the second set of light source being configured to illuminate the right eye with the light after the light passes through the second lens. ### Claim 17 (depends on 16) The apparatus of claim 16, wherein the controller is further configured to command a first one of the first set of light sources to output the light. ### Claim 18 (depends on 17) The apparatus of claim 17, the controller being configured to stop the light output from the first one of the plurality of light sources comprises: the controller being configured to stop the light output from the first one of the first set of light sources. ### Claim 19 (depends on 18) The apparatus of claim 18, the controller further being configured to command one different from the first one of the first set of light sources to output the light if the light output from the first one of the first set of light sources is stopped. ### Claim at position 20 20-27. (canceled) ### Claim 28 A method comprising: generating, by a controller, a first bitmap representing reflections in a first image frame captured by an image capture unit within a stereoscopic image viewer; generating, by the controller, a second bitmap representing reflections in a second image frame captured by the image capture unit within the stereoscopic image viewer; subtracting, by the controller, the second bitmap from the first bitmap to obtain a new bitmap; and stopping, by the controller, light output from one of a plurality of light sources within the stereoscopic image viewer if all bits in the new bitmap have a same state. ### Claim 29 (depends on 28) The method of claim 28, further comprising: commanding, by the controller, a different one of the plurality of light sources to output light if the light output from the one of the plurality of light sources is stopped. ### Claim 30 (depends on 28) The method of claim 28, further comprising: commanding, by the controller, the one of the plurality of light sources to output light prior to the stopping. ### Claim 31 (depends on 1) The apparatus of claim 1, wherein at least one light source included in the plurality of light sources is provided on a user side of the viewing lens.