# US 12,599,498 B2 · Automated image guidance for ophthalmic surgery > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,599,498 B2 - Document ID: US12599498B2 - Status: Granted - Kind: B2 - Application number: 18/463,121 - Filed: 2023-09-07 - Published: 2024-03-14 - Granted: 2026-04-14 - Earliest claimed: 2022-09-12 - Assignees: Alcon Inc. - Applicants: Alcon Inc. - Inventors: Tobias Jura Rapoport - Web page: https://patentgrep.com/patent/US12599498B2 ## CPC - A61B 3/0058: {for multiple images} - A61F 9/007: Methods or devices for eye surgery - A61B 3/13: Ophthalmic microscopes - A61B 3/14: Arrangements specially adapted for eye photography - A61B 90/20: Surgical microscopes characterised by non-optical aspects - G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)} - G06T 7/33: using feature-based methods - G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) - G06V 40/193: {Preprocessing; Feature extraction} - H04N 5/265: Mixing - A61B 2034/2065: {Tracking using image or pattern recognition} - A61B 2090/365: {augmented reality, i.e. correlating a live optical image with another image} - A61F 9/00736: {Instruments for removal of intra-ocular material or intra-ocular injection, e.g. cataract instruments (A61F9/008 takes precedence)} - G06N 20/00: Machine learning - G06T 2207/10056: Microscopic image - G06T 2207/30041: Eye; Retina; Ophthalmic - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - A61F: FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS (dental prosthetics A61C) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract In certain embodiments, an ophthalmic system and computer-implemented method for automatically initializing an image guided surgery are described. The initialization includes monitoring a scene using multiple images captured by a first imaging device. An eye of a user is detected within a first image of the multiple images. In response to detecting the eye of the user, a registration procedure is initiated with the first image and a reference image of the eye of the user to generate a set of transformation information. A set of overlay content is generated based on the set of transformation information. The set of overlay content includes a transformed first image or a transformed reference image. Overlay content is presented onto the scene via a second imaging device. ## Claims ### Claim 1 An ophthalmic system comprising: a first imaging device adapted to capture a plurality of images of a scene within an intra-operative environment; a second imaging device adapted to visualize the scene; a memory comprising executable instructions; and a processor in data communication with the memory and configured to execute the executable instructions to: monitor the scene using the plurality of images captured by the first imaging device; upon detecting an eye of a user within a first image of the plurality of images, automatically initialize an image-guided surgery, comprising initiating a registration procedure to generate a set of transformation information, based at least in part on the first image and a reference image of the eye of the user; generate a set of overlay content based on the set of transformation information, wherein the set of overlay content comprises (i) a transformed first image or (ii) a transformed reference image; and present the overlay content onto the scene via the second imaging device. ### Claim 2 (depends on 1) The ophthalmic system of claim 1, wherein the processor is further configured to execute the executable instructions to provide feedback on a position of the eye, based on the monitoring. ### Claim 3 (depends on 2) The ophthalmic system of claim 2, wherein the feedback is provided while the plurality of images are being captured by the first imaging device. ### Claim 4 (depends on 1) The ophthalmic system of claim 1, wherein the transformation information comprises at least one of: (i) scaling information, (ii) translation information, or (iii) rotation information. ### Claim 5 (depends on 1) The ophthalmic system of claim 1, wherein detecting the eye of the user comprises detecting, via an eye detection algorithm, a set of features within the first image associated with ocular structures. ### Claim 6 (depends on 5) The ophthalmic system of claim 5, wherein the set of features comprises at least one of a speculum or a limbus. ### Claim 7 (depends on 1) The ophthalmic system of claim 1, wherein the processor is further configured to execute the executable instructions to validate the set of transformation information prior to generation of the overlay content. ### Claim 8 (depends on 1) The ophthalmic system of claim 1, wherein: the first image is an intra-operative image; and the reference image is a pre-operative image. ### Claim 9 A computer-implemented method comprising: monitoring a scene in an intra-operative environment using a plurality of images captured by a first imaging device; upon detecting an eye of a user within a first image of the plurality of images, automatically initializing an image-guided surgery, wherein automatically initializing the image-guided surgery comprises initiating a registration procedure to generate a set of transformation information, based at least in part on the first image and a reference image of the eye of the user; generating a set of overlay content based on the set of transformation information, wherein the set of overlay content comprises (i) a transformed first image or (ii) a transformed reference image; and presenting the overlay content onto the scene via a second imaging device. ### Claim 10 (depends on 9) The computer-implemented method of claim 9, further comprising providing feedback on a position of the eye, based on the monitoring. ### Claim 11 (depends on 10) The computer-implemented method of claim 10, wherein the feedback is provided while the plurality of images are being captured by the first imaging device. ### Claim 12 (depends on 9) The computer-implemented method of claim 9, wherein the transformation information comprises at least one of: (i) scaling information, (ii) translation information, or (iii) rotation information. ### Claim 13 (depends on 9) The computer-implemented method of claim 9, wherein detecting the eye of the user comprises detecting, via an eye detection algorithm, a set of features within the first image associated with ocular structures. ### Claim 14 (depends on 13) The computer-implemented method of claim 13, wherein the set of features comprises at least one of a speculum or a limbus. ### Claim 15 (depends on 9) The computer-implemented method of claim 9, further comprising validating the set of transformation information prior to generation of the overlay content. ### Claim 16 (depends on 9) The computer-implemented method of claim 9, wherein: the first image is an intra-operative image; and the reference image is a pre-operative image. ### Claim 17 A non-transitory computer-readable medium having computer executable instructions stored thereon, the computer executable instructions being executable by one or more processors to perform an operation comprising: monitoring a scene in an intra-operative environment using a plurality of images captured by a first imaging device; upon detecting an eye of a user within a first image of the plurality of images, automatically initializing an image-guided surgery, wherein automatically initializing the image-guided surgery comprises initiating a registration procedure to generate a set of transformation information, based at least in part on the first image and a reference image of the eye of the user; generating a set of overlay content based on the set of transformation information, wherein the set of overlay content comprises (i) a transformed first image or (ii) a transformed reference image; and presenting the overlay content onto the scene via a second imaging device. ### Claim 18 (depends on 17) The non-transitory computer-readable medium of claim 17, the operation further comprising providing feedback on a position of the eye, based on the monitoring. ### Claim 19 (depends on 18) The non-transitory computer-readable medium of claim 18, wherein the feedback is provided while the plurality of images are being captured by the first imaging device. ### Claim 20 (depends on 17) The non-transitory computer-readable medium of claim 17, wherein: detecting the eye of the user comprises detecting, via an eye detection algorithm, a set of features within the first image associated with ocular structures; and the set of features comprises at least one of a speculum or a limbus.