# US 2018/0024341 A1 · Augmented stereoscopic microscopy

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0024341 A1
- Document ID: US20180024341A1
- Status: Published application
- Kind: A1
- Application number: 15/548,807
- Filed: 2016-02-05
- Published: 2018-01-25
- Earliest claimed: 2015-02-09
- Assignees: Arizona Board of Regents on Behalf of the University of Arizona
- Applicants: Arizona Board of Regents on Behalf of the University of Arizona
- Inventors: MAREK ROMANOWSKI; JEFFREY WATSON; CHRISTIAN GAINER
- Web page: https://patentgrep.com/patent/US20180024341A1

## CPC

- A61B 1/000094: {extracting biological structures}
- G02B 21/0012: {Surgical microscopes (counterbalanced structures for surgical microscopes G02B7/001)}
- G02B 21/02: Objectives
- G02B 21/06: Means for illuminating specimens
- G02B 21/16: adapted for ultraviolet illumination {; Fluorescence microscopes (G02B21/0076 takes precedence)}
- G02B 21/22: Stereoscopic arrangements
- G02B 21/361: {Optical details, e.g. image relay to the camera or image sensor (G02B21/364 takes precedence; illumination details G02B21/06 and subgroups)}
- G02B 21/365: {Control or image processing arrangements for digital or video microscopes (G02B21/361, G02B21/362 take precedence)}
- A61B 1/0005: {combining images e.g. side-by-side, superimposed or tiled}
- A61B 1/3132: {for laparoscopy}
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS

## Abstract

Various examples are provided related to imaging with augmented stereoscopic microscopes. In one example, an augmented stereoscopic microscope includes an objective lens that can simultaneously receive near infrared (NIR) images and visible bright-field images of an examined object and an augmentation module. The augmentation module can separate the NIR images from the visible bright-field images for processing by an image processing unit to produce synthetic images using the NIR images and combine the synthetic images with the visible bright-field images to form co-registered augmented images that are directed to an eyepiece of the augmented stereoscopic microscope. In another example, a method includes obtaining a NIR image of an examined object; generating a synthetic image using the NIR image; combining the synthetic image with a real-time visual image of the examined object to form an augmented image; and directing the augmented image to an eyepiece of an augmented stereoscopic microscope.

## Claims

### Claim 1

An augmented stereoscopic microscope, comprising:
an objective lens configured to simultaneously receive near infrared (NIR) images and visible bright-field images of an examined object; and
an augmentation module configured to:
separate the NIR images from the visible bright-field images for processing by an image processing unit to produce synthetic images based upon the NIR images; and
combine the synthetic images with the visible bright-field images to form co-registered augmented images that are directed to an eyepiece of the augmented stereoscopic microscope.

### Claim 2 (depends on 1)

The augmented stereoscopic microscope of claim 1, wherein the augmentation module comprises a beamsplitter configured to combine the synthetic images with the visible bright-field images to form the co-registered augmented images.

### Claim 3 (depends on 2)

The augmented stereoscopic microscope of claim 2, wherein combining the synthetic images with the visible bright-field images comprises projecting the synthetic images onto the beamsplitter to overlay the visible bright-field images.

### Claim 4 (depends on 2)

The augmented stereoscopic microscope of claim 2, wherein the beamsplitter is a dichroic beamsplitter.

### Claim 5 (depends on 1)

The augmented stereoscopic microscope of claim 1, wherein the augmentation module comprises a spatial light modulator configured to combine the synthetic images with the visible bright-field images to form the co-registered augmented images.

### Claim 6 (depends on 5)

The augmented stereoscopic microscope of claim 5, wherein the spatial light modulator is a micromirror array or a switchable mirror.

### Claim 7 (depends on 6)

The augmented stereoscopic microscope of claim 6, wherein combining the synthetic images with the visible bright-field images comprises switching at least a portion of the micromirror array to reflect at least a portion of the synthetic images and at least a portion of the visible bright-field images.

### Claim 8 (depends on 1)

The augmented stereoscopic microscope of claim 1, wherein the synthetic images comprise visible pseudo-color images.

### Claim 9 (depends on 1)

The augmented stereoscopic microscope of claim 1, wherein the augmentation module comprises a light emitting diode (LED) configured to illuminate the examined object with NIR excitation light via the objective lens.

### Claim 10 (depends on 1)

The augmented stereoscopic microscope of claim 1, wherein the augmentation module comprises a sensor configured to capture the NIR images.

### Claim 11 (depends on 10)

The augmented stereoscopic microscope of claim 10, wherein the sensor is further configured to capture the co-registered augmented images.

### Claim 12 (depends on 1)

The augmented stereoscopic microscope of claim 1, wherein the augmentation module comprises a sensor configured to capture the co-registered augmented images.

### Claim 13 (depends on 1)

The augmented stereoscopic microscope of claim 1, wherein the augmentation module comprises a short-pass filter configured to remove NIR light from the co-registered augmented images.

### Claim 14 (depends on 1)

The augmented stereoscopic microscope of claim 1, wherein the augmentation module is configured to form co-registered augmented images for a right optical path of the augmented stereoscopic microscope and co-registered augmented images for a left optical path of the augmented stereoscopic microscope.

### Claim 15 (depends on 1)

The augmented stereoscopic microscope of claim 1, wherein the synthetic images comprise prerecorded information associated with the examined object.

### Claim 16 (depends on 15)

The augmented stereoscopic microscope of claim 15, wherein the prerecorded information is obtained through magnetic resonance imaging (MRI), computed tomography (CT), or positron emission tomography (PET) of the examined object.

### Claim 17

A method, comprising:
obtaining a near infrared (NIR) image of an examined object;
generating a synthetic image based upon the NIR image;
combining the synthetic image with a real-time visual image of the examined object to form an augmented image; and
directing the augmented image to an eyepiece of an augmented stereoscopic microscope.

### Claim 18 (depends on 17)

The method of claim 17, wherein combining the synthetic image with the real-time visual image comprises projecting the synthetic image onto a beamsplitter to overlay the real-time visual image.

### Claim 19 (depends on 17)

The method of any of claim 17, wherein combining the synthetic image with the real-time visual image comprises switching at least a portion of a micromirror array to reflect at least a portion of the synthetic image and at least a portion of the real-time visual image.

### Claim 20 (depends on 17)

The method of claim 17, comprising illuminating the examined object with NIR excitation through an objective lens of the augmented stereoscopic microscope.

### Claim 21 (depends on 17)

The method of claim 17, wherein the examined object comprises a NIR laser.

### Claim 22 (depends on 17)

The method of claim 17, wherein the synthetic image comprises a chemical concentration in the examined object.

### Claim 23 (depends on 17)

The method of claim 17, wherein the synthetic image comprises a temperature of the examined object.
