# US 10,481,403 B2 · Contact lens with retinal camera > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,481,403 B2 - Document ID: US10481403B2 - Status: Granted - Kind: B2 - Application number: 15/898,157 - Filed: 2018-02-15 - Published: 2019-08-15 - Granted: 2019-11-19 - Earliest claimed: 2018-02-15 - Assignees: Tectus Corporation - Applicants: Spy Eye, LLC - Inventors: Paul Scott Martin - Web page: https://patentgrep.com/patent/US10481403B2 ## CPC - G02B 27/0172: {characterised by optical features} - A61B 3/125: with contact lenses - G02B 13/16: for use in conjunction with image converters or intensifiers {, or for use with projectors, e.g. objectives for projection TV} - G02B 17/061: {on-axis systems with at least one of the mirrors having a central aperture} - G02B 27/017: {Head mounted} - G02B 27/0179: {Display position adjusting means not related to the information to be displayed} - G02C 7/04: Contact lenses for the eyes (disinfection or sterilisation of contact lenses A61L12/00) - G02C 7/049: {Contact lenses having special fitting or structural features achieved by special materials or material structures} - G02C 11/10: {Electronic devices other than hearing aids} - H04N 23/00: Cameras or camera modules comprising electronic image sensors; Control thereof - H04N 23/60: Control of cameras or camera modules - H04N 23/63: by using electronic viewfinders - A61B 3/113: for determining or recording eye movement - G02B 2027/0138: {comprising image capture systems, e.g. camera} - G02B 2027/014: {comprising information/image processing systems} - G02B 2027/0187: {slaved to motion of at least a part of the body of the user, e.g. head, eye} - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G02C: SPECTACLES; SUNGLASSES OR GOGGLES INSOFAR AS THEY HAVE THE SAME FEATURES AS SPECTACLES; CONTACT LENSES (trial frames for testing the eyes A61B3/04; goggles or eyeshields not having the same features as spectacles A61F9/00) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A contact lens contains an inward pointing camera, which will be referred to as a retinal camera since it images light reflected from the retina. These can be reflections of physical features of the retina, of images of an external scene imaged by the eye onto the retina, or images projected onto the retina for example from small projectors contained in the contact lens (femtoprojectors). The field of view (FOV) of the retinal camera is sufficiently large that these reflections can be tracked relative to each other and/or relative to their position within the retinal camera's FOV. This information can be processed to track eye gaze and movement relative to the outside world, to align images from the femtoprojector with the eye and/or to align images from the femtoprojector with images from the outside world, among other tasks. ## Claims ### Claim 1 A system comprising: an eye-mounted device comprising: a scleral contact lens that sits on a sclera of a wearer's eye; and a retinal camera contained in the scleral contact lens, the retinal camera imaging light reflected from a retina of a wearer of the scleral contact lens, the reflected light imaged by the retinal camera comprising a reflection of an external scene imaged by the eye onto the retina; and a processing system that determines a gaze direction of the eye relative to the external scene based on the reflected light imaged by the retinal camera. ### Claim 2 (depends on 1) The system of claim 1 where: the reflected light imaged by the retinal camera further comprises a reflection of retinal features of the eye; and the processing system determines the gaze direction based on a relative positioning between the reflection of the retinal features and the reflection of the external scene. ### Claim 3 (depends on 1) The system of claim 1 where: the processing system determines the gaze direction based (a) on a location of the reflection of the external scene within a field of view of the retinal camera, and also (b) on a known location of the field of view of the retinal camera on the retina of the eye. ### Claim 4 (depends on 1) The system of claim 1 further comprising: a femtoprojector also contained in the scleral contact lens, the femtoprojector performing a projection of images onto a projection area on the retina; where: the reflected light imaged by the retinal camera further comprises a reflection of the projected image; and the processing system determines the gaze direction based (a) on a relative positioning between the reflection of the external scene and the reflection of the projected image, and also (b) on a known location of the projected image on the retina of the eye. ### Claim 5 (depends on 1) The system of claim 1 further comprising: a femtoprojector also contained in the scleral contact lens, the femtoprojector performing a projection of images onto a projection area on the retina, where a location of an object in the projected image depends on the gaze direction. ### Claim 6 (depends on 1) The system of claim 1 where the retinal camera images a sequence of frames of light reflected from the retina; and the processing system determines the gaze direction based on temporal differences between different frames. ### Claim 7 (depends on 1) The system of claim 1 where the gaze direction is used in real-time operation of the eye-mounted device. ### Claim 8 (depends on 1) The system of claim 1 further comprising: a femtoprojector also contained in the scleral contact lens, the femtoprojector performing a projection of images onto a projection area on the retina. ### Claim 9 (depends on 8) The system of claim 8 further comprising: a processing system that determines a relative positioning based on the reflected light imaged by the retinal camera, the relative positioning used to determine a location of the projection. ### Claim 10 (depends on 9) The system of claim 9 where the relative positioning is used to determine the location of the projection on the retina. ### Claim 11 (depends on 10) The system of claim 10 where: the reflected light imaged by the retinal camera comprises a reflection of the projected image and a reflection of retinal features of the eye; the relative positioning determined by the processing system is relative positioning between the reflection of the projected image and the reflection of the retinal features; and the location of the projection on the retina is determined based on the relative positioning. ### Claim 12 (depends on 10) The system of claim 10 where: the reflected light imaged by the retinal camera comprises a reflection of retinal features of the eye; the relative positioning determined by the processing system is a location of the reflection of the retinal features within the field of view of the retinal camera; and the location of the projection on the retina is determined based (a) on a known location of the projection area relative to a field of view of the retinal camera, and also (b) on the location of the reflection of the retinal features within the field of view of the retinal camera. ### Claim 13 (depends on 10) The system of claim 10 where: the reflected light imaged by the retinal camera comprises a reflection of the projected image; the relative positioning determined by the processing system is a location of the reflection of the projected image within a field of view of the retinal camera; and the location of the projection on the retina is determined based (a) on the location of the reflection of the projected image within a field of view of the retinal camera, and also (b) on a known location of the field of view of the retinal camera on the retina of the eye. ### Claim 14 (depends on 9) The system of claim 9 where the relative positioning is used to determine a location of the projection relative to an external scene imaged by the eye onto the retina. ### Claim 15 (depends on 14) The system of claim 14 where: the reflected light imaged by the retinal camera comprises a reflection of the projected image and a reflection of the external scene imaged onto the retina; the relative positioning determined by the processing system is relative positioning between the reflection of the projected image and the reflection of the external scene; and the location of the projection relative to the external scene is determined based on the relative positioning. ### Claim 16 (depends on 14) The system of claim 14 where: the reflected light imaged by the retinal camera comprises a reflection of the external scene imaged onto the retina; the relative positioning determined by the processing system is a location of the reflection of the external scene within the field of view of the retinal camera; and the location of the projection relative to the external scene is determined based (a) on a known location of the projection area relative to a field of view of the retinal camera, and also (b) on the location of the reflection of the external scene within the field of view of the retinal camera. ### Claim 17 (depends on 9) The system of claim 9 where the femtoprojector projects a calibration image onto the retina; the reflected light imaged by the retinal camera comprises a reflection of the calibration image; and the relative positioning is determined based on a location of the reflection of the calibration image within a field of view of the retinal camera.