# US 9,775,515 B2 · System and method for multi-scale closed-loop eye tracking with real-time image montaging > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,775,515 B2 - Document ID: US9775515B2 - Status: Granted - Kind: B2 - Application number: 15/158,803 - Filed: 2016-05-19 - Published: 2016-12-01 - Granted: 2017-10-03 - Earliest claimed: 2015-05-28 - Assignees: University of Rochester - Applicants: Qiang Yang; Jie Zhang - Inventors: Qiang Yang; Jie Zhang - Web page: https://patentgrep.com/patent/US9775515B2 ## CPC - A61B 3/152: {for aligning} - A61B 3/0025: {characterised by electronic signal processing, e.g. eye models} - A61B 3/113: for determining or recording eye movement - A61B 3/1225: {using coherent radiation} - G02B 26/105: {with one or more pivoting mirrors or galvano-mirrors (G02B26/101 takes precedence)} - G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking} - G02B 27/646: {compensating for small deviations, e.g. due to vibration or shake (movement of one or more optical elements for control of motion blur in cameras, projectors or printers G03B2205/0007; image stabilisation in cameras peculiar to the presence or use of an electronic image sensor H04N23/68)} - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS ## Abstract A system for multi-scale closed-loop eye tracking to compensate for translation and rotation motion while imaging in vivo a surface area of an internal structure of an eye of a subject includes a narrow field imaging device optically coupled to an optical path to receive light reflected from the surface area of the structure of the eye. A wide field camera is optically coupled to the optical path by a beam splitter disposed in the optical path. A tracking mirror is disposed in the optical path between the beam splitter and the structure of the eye. A control process algorithm actively compensates substantially in real time for both translational and rotational movements of the eye. A system where a torsional correction device causes a rotating movement of a subject's head and a method for multi-scale closed-loop eye tracking are also described. ## Claims ### Claim 1 A system for multi-scale closed-loop eye tracking to compensate for translation and rotation motion while imaging in vivo a surface area of an internal structure of an eye of a subject comprising: a narrow field imaging device optically coupled to an optical path to receive light reflected from the surface area of the structure of the eye; a wide field camera optically coupled to the optical path by a beam splitter disposed in said optical path; a tracking mirror disposed in said optical path between said beam splitter and the structure of the eye; a torsional correction device mechanically coupled to one or more optical components of the system; a control process algorithm running on a computer, said computer communicatively coupled to said wide field camera and said narrow field imaging device and said tracking mirror and said torsional correction device, and wherein said control process algorithm causes movements of said tracking mirror and said torsional correction device to actively compensate substantially in real time for both translational and rotational movements of the eye at least in part based on feedback images from said wide field camera and said narrow field imaging device; and an additional steering mirror disposed in said optical path between said narrow field imaging device and said beam splitter, said additional steering mirror communicatively coupled to said computer and controlled by said control process algorithm to provide an additional translational correction. ### Claim 2 (depends on 1) The system for multi-scale closed-loop eye tracking of claim 1, wherein said wide field camera comprises a Fundus camera. ### Claim 3 (depends on 1) The system for multi-scale closed-loop eye tracking of claim 1, wherein said narrow field imaging device comprises an AOSLO imaging apparatus. ### Claim 4 (depends on 3) The system for multi-scale closed-loop eye tracking of claim 3, wherein said AOSLO imaging apparatus is optically turned off when an AOSLO scanner runs out of an imaging FOV. ### Claim 5 (depends on 3) The system for multi-scale closed-loop eye tracking of claim 3, further comprising an over-sampling analog to digital converter (A/D) in combination with a pixel-binning process algorithm which runs on a pixel-binning hardware to increase a signal to noise ratio (SNR) of a raw image from said AOSLO imaging apparatus. ### Claim 6 (depends on 1) The system for multi-scale closed-loop eye tracking of claim 1, wherein said tracking mirror comprises at least one or more galvano scanning mirrors. ### Claim 7 (depends on 1) The system for multi-scale closed-loop eye tracking of claim 1, wherein said wide field camera and said narrow field imaging device are mounted on a rotational stage mechanically coupled to said torsional correction device. ### Claim 8 (depends on 1) The system for multi-scale closed-loop eye tracking of claim 1, wherein either of said wide field camera or said narrow field imaging device, is mounted on a rotational stage mechanically coupled to said torsional correction device. ### Claim 9 (depends on 8) The system for multi-scale closed-loop eye tracking of claim 8, wherein said torsional correction device comprises a motor. ### Claim 10 (depends on 1) The system for multi-scale closed-loop eye tracking of claim 1, wherein said system comprises an integration of multiple channels of data I/O on a single personal computer (PC). ### Claim 11 A system for multi-scale closed-loop eye tracking to compensate for translation and rotation motion while imaging in vivo a surface area of an internal structure of an eye of a subject where a subject's head is supported by the system comprising: a narrow field imaging device optically coupled to an optical path to receive light reflected from the surface area of a structure of the eye; a wide field camera optically coupled to the optical path by a beam splitter disposed in said optical path; a tracking mirror disposed in said optical path between said beam splitter and the structure of the eye; a torsional correction device comprising a rotating mount which rotates the subject's head in a plane about parallel to a surface of the surface area of the structure of the eye, said rotating mount comprising a chin rest at a first end of said rotating mount, said rotating mount extending from rearward from said first end of said rotating mount to a second end of said rotating mount behind the subject's head, and a motor mechanically and rotatingly coupled said second end of said rotating mount; and a control process algorithm running on a computer, said computer communicatively coupled to said wide field camera and said narrow field imaging device and said tracking mirror and said torsional correction device rotating mount, and wherein said control process algorithm causes movements of said tracking mirror and said motor of said torsional correction device rotating mount to actively compensate substantially in real time for both translational and rotational movements of the eye, at least in part based on feedback images from said wide field camera and said narrow field imaging device. ### Claim 12 A method for multi-scale closed-loop eye tracking to compensate for translation and rotation motion while imaging in vivo a surface area of an internal structure of an eye of a subject's head comprising: providing a narrow field imaging device optically coupled to an optical path to receive light reflected from the surface area of the structure of the eye, a wide field camera optically coupled to the optical path by a beam splitter disposed in said optical path, a tracking mirror disposed in said optical path between said beam splitter and the structure of the eye, a torsional correction device, and a control process algorithm running on a computer; calculating by computer a translation and a rotation of the eye at least in part from an image received from said wide field camera and said narrow field imaging device; and setting by computer a position of said tracking mirror to compensate for said translation of the eye and setting by computer a rotational movement of said torsional correction device, to compensate for said rotation of the eye, and setting by computer said torsional correction device which rotates both of said wide field camera and said narrow field imaging device to compensate for said rotation of the eye. ### Claim 13 A system for multi-scale closed-loop eye tracking to compensate for translation and rotation motion while imaging in vivo a surface area of an internal structure of an eye of a subject comprising: a narrow field imaging device optically coupled to an optical path to receive light reflected from the surface area of the structure of the eye; a wide field camera optically coupled to the optical path by a beam splitter disposed in said optical path; a tracking mirror disposed in said optical path between said beam splitter and the structure of the eye; a torsional correction device mechanically coupled to one or more optical components of the system; a control process algorithm running on a computer, said computer communicatively coupled to said wide field camera and said narrow field imaging device and said tracking mirror and said torsional correction device, and wherein said control process algorithm causes movements of said tracking mirror and said torsional correction device to actively compensate substantially in real time for both translational and rotational movements of the eye at least in part based on feedback images from said wide field camera and said narrow field imaging device; and an over-sampling analog to digital converter (A/D) in combination with a pixel-binning process algorithm which runs on a pixel-binning hardware to increase a signal to noise ratio (SNR) of a raw image from said AOSLO imaging apparatus. ### Claim 14 (depends on 13) The system for multi-scale closed-loop eye tracking of claim 13, wherein said wide field camera comprises a Fundus camera. ### Claim 15 (depends on 13) The system for multi-scale closed-loop eye tracking of claim 13, wherein said narrow field imaging device comprises an AOSLO imaging apparatus. ### Claim 16 (depends on 15) The system for multi-scale closed-loop eye tracking of claim 15, wherein said AOSLO imaging apparatus is optically turned off when an AOSLO scanner runs out of an imaging FOV. ### Claim 17 (depends on 13) The system for multi-scale closed-loop eye tracking of claim 13, wherein said tracking mirror comprises at least one or more galvano scanning mirrors. ### Claim 18 (depends on 13) The system for multi-scale closed-loop eye tracking of claim 13, wherein said wide field camera and said narrow field imaging device are mounted on a rotational stage mechanically coupled to said torsional correction device. ### Claim 19 (depends on 13) The system for multi-scale closed-loop eye tracking of claim 13, wherein either of said wide field camera or said narrow field imaging device, is mounted on a rotational stage mechanically coupled to said torsional correction device. ### Claim 20 (depends on 19) The system for multi-scale closed-loop eye tracking of claim 19, wherein said torsional correction device comprises a motor. ### Claim 21 (depends on 13) The system for multi-scale closed-loop eye tracking of claim 13, wherein said system comprises an integration of multiple channels of data I/O on a single personal computer (PC).