# US 2019/0377170 A1 · Method and apparatus for imaging unsectioned tissue specimens

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0377170 A1
- Document ID: US20190377170A1
- Status: Published application
- Kind: A1
- Application number: 16/531,714
- Filed: 2019-08-05
- Published: 2019-12-12
- Earliest claimed: 2016-02-12
- Assignees: Massachusetts Institute of Technology
- Applicants: Massachusetts Institute of Technology
- Inventors: James G. Fujimoto; Michael G. Giacomelli; Tadayuki Yoshitake; Lucas C. Cahill
- Web page: https://patentgrep.com/patent/US20190377170A1

## CPC

- G02B 21/367: {providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison}
- G01N 21/01: Arrangements or apparatus for facilitating the optical investigation
- G01N 21/6428: {Measuring fluorescence of fluorescent products of reactions or of fluorochrome labelled reactive substances, e.g. measuring quenching effects, using measuring "optrodes" (in vivo A61B5/00; immunoassay G01N33/53)}
- G01N 21/6458: {Fluorescence microscopy (fluorescence microscopes per se G02B21/0076 and G02B21/16)}
- G01N 21/6486: {Measuring fluorescence of biological material, e.g. DNA, RNA, cells (G01N21/6428 takes precedence)}
- G02B 21/0088: {Inverse microscopes}
- G02B 21/086: {for transillumination only}
- G02B 21/16: adapted for ultraviolet illumination {; Fluorescence microscopes (G02B21/0076 takes precedence)}
- G02B 21/26: Stages; Adjusting means therefor
- G02B 21/361: {Optical details, e.g. image relay to the camera or image sensor (G02B21/364 takes precedence; illumination details G02B21/06 and subgroups)}
- G01N 21/6408: {with measurement of decay time, time resolved fluorescence}
- G01N 2021/0106: {General arrangement of respective parts}
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G01N: INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES (measuring or testing apparatus or processes other than immunoassay, involving enzymes or microorganisms C12M, C12Q)

## Abstract

An apparatus and method for real-time optical imaging of a tissue specimen. The apparatus comprises a primary imaging system configured to use an illumination source to acquire images of a tissue specimen through one or more spectrally separated channels, and configured to perform optical depth sectioning; an auxiliary imaging system, configured to acquire an auxiliary image of the tissue specimen; a specimen holder having a transparent window therewithin, window, the specimen holder comprising one or more position sensors, wherein the specimen holder is configured to be translatable in the specimen plane; a user input device configured to accept user input, wherein the specimen holder is configured to translate in response to the user input in real-time; a processing unit configured to execute a sequence of instructions on the sequence of images acquired by the primary imaging system, the auxiliary image, and at least one specimen holder position to generate a composite representation of the tissue specimen that includes a representation of cell nuclei in the specimen; and a display device configured to display the composite representation of the tissue specimen in real-time.

## Claims

### Claim 1

An apparatus for real-time optical imaging of a tissue specimen, the apparatus comprising:
a primary imaging system configured to use an illumination source to acquire images of a tissue specimen through one or more spectrally separated channels, at least one of the one or more spectrally separated channels is configured to detect a range of wavelengths distinct from the range of wavelengths of the illumination source, the primary imaging system being an inverted microscope having a frame acquisition rate and configured to perform optical depth sectioning, the primary image system being configured to acquire a sequence of images;
an auxiliary imaging system, wherein the auxiliary imaging system is configured to acquire an auxiliary image of the tissue specimen, wherein the area of the auxiliary image is greater than the area of each image of the sequence of images acquired by the primary imaging system;
a specimen holder having a transparent window therewithin, the specimen holder being disposed in a specimen plane at a focal plane of the primary imaging system, the specimen holder being configured to hold the tissue specimen on the transparent window, the specimen holder comprising one or more position sensors, wherein the specimen holder is configured to be translatable in the specimen plane, the one or more position sensors being configured to measure a specimen holder position, and wherein the specimen holder is configured to be translatable to a focal plane of the auxiliary imaging system;
wherein the primary imaging system and the auxiliary imaging system are both disposed on the same side of the specimen plane, the apparatus further comprising:
a user input device configured to accept user input, wherein the specimen holder is configured to translate in response to the user input in real-time;
a processing unit in electrical communication with the primary imaging system, the auxiliary imaging system, and the position sensors, wherein the processing unit is configured to execute a sequence of instructions on the sequence of images acquired by the primary imaging system, the auxiliary image, and at least one specimen holder position to generate a composite representation of the tissue specimen that includes a representation of cell nuclei in the specimen; and
a display device in electrical communication with the processing unit, the display device being configured to display the composite representation of the tissue specimen in real-time.

### Claim 2 (depends on 1)

The apparatus of claim 1 wherein the representation of cell nuclei comprises virtual transillumination microscopy.

### Claim 3 (depends on 2)

The apparatus of claim 2 wherein the primary imaging system is a multiphoton, confocal, light sheet, ultraviolet surface excitation, or structured illumination microscope.

### Claim 4 (depends on 2)

The apparatus of claim 2, wherein each image of the sequence of images acquired by the primary imaging system is acquired through two or more spectrally separated channels;
and wherein at least one of the two or more spectrally separated channels is configured to detect light emitted by a nuclear contrast agent, and wherein at least one of the two or more spectrally separated channels is configured to detect light emitted from a complementary source of contrast.

### Claim 5 (depends on 2)

The apparatus of claim 2 wherein the specimen holder further comprises one or more actuators for translation of the specimen holder.

### Claim 6 (depends on 2)

The apparatus of claim 2 wherein the user input device incorporates mechanical transmission of force from the user to the specimen holder to move the specimen holder.

### Claim 7 (depends on 2)

The apparatus of claim 2 wherein the auxiliary imaging system comprises a one dimensional digital array of photosensors.

### Claim 8 (depends on 2)

The apparatus of claim 2 wherein the auxiliary imaging system comprises two or more spectrally distinct channels.

### Claim 9 (depends on 8)

The apparatus of claim 8, wherein the processing unit is configured to compare the two or more spectrally distinct channels of the auxiliary imaging system to identify the location of the surgical inks, sutures or exogenous markings in the tissue samples.

### Claim 10 (depends on 9)

The apparatus of claim 9, wherein the composite representation of the tissue specimen includes one or more locations of the surgical inks, sutures or exogenous markings on the tissue specimen.

### Claim 11 (depends on 2)

The apparatus of claim 2 wherein the auxiliary imaging system includes a narrow band illuminator.

### Claim 12 (depends on 2)

The apparatus of claim 2 wherein the specimen holder further includes a removable transparent window.

### Claim 13 (depends on 12)

The apparatus of claim 12 wherein the removable transparent window incorporates a fluid retaining structure surrounding the transparent window.

### Claim 14 (depends on 2)

The apparatus of claim 2 wherein the specimen holder further includes one or more of a specimen divider or a lid.

### Claim 15 (depends on 14)

The apparatus of claim 14 wherein the auxiliary imaging system further includes an illumination source.

### Claim 16 (depends on 15)

The apparatus of claim 15 wherein the illumination source is configured to be attached to the lid.

### Claim 17 (depends on 19)

The apparatus of claim 19 wherein the lid further includes a specimen guide configured to hold the tissue specimen against the transparent window.

### Claim 18 (depends on 2)

The apparatus of claim 2 wherein the processing unit further includes a graphics processing unit having hardware for parallel processing, and further wherein the graphics processing unit includes computer executable instructions for a series of parallel processing operations on the sequence of images acquired by the primary imaging system.

### Claim 19 (depends on 2)

The apparatus of claim 2 wherein the processing unit is configured to update the composite representation of the tissue specimen after translation of the specimen holder in less than 250 milliseconds.

### Claim 20 (depends on 18)

The apparatus of claim 18, wherein the graphics processing unit includes computer executable instructions for a series of parallel processing operations for virtual transillumination microscopy processing.

### Claim 21 (depends on 4)

The apparatus of claim 4 wherein the processing unit includes computer executable instructions for virtual transillumination microscopy processing on the sequence of images using the nonlinear process:
R=(1/(1−exp(−k)){circumflex over ( )}2)*(exp(−B_Hematoxylin,red*I_nuclear*k)−exp(−k))*(exp(−B_Eosin,red*I_secondary*k)−exp(−k))
G=(1/(1−exp(−k)){circumflex over ( )}2)*(exp(−B_Hematoxylin,green*I_nuclear*k)−exp(−k))*(exp(−B_Eosin,green*I_secondary*k)−exp(−k))
B=(1/(1−exp(−k)){circumflex over ( )}2)*(exp(−B_Hematoxylin,blue*I_nuclear*k)−exp(−k))*(exp(−B_Eosin,blue*I_secondary*k)−exp(−k))
where R, G, and B are the red, green and blue intensities of the virtual transillumination images respectively, B_Hematoxylin,red, B_Hematoxylin,green and B_Hematoxylin,blue are the absorption of hematoxylin for red, green and blue light respectively, B_Eosin,red, B_Eosin,green, and B_Eosin,blue are the absorption of eosin for red, green and blue light respectively, while I_nuclear is the intensity of the contrast agent has specificity for cell nuclei or components of cell nuclei, I_secondary is the intensity of the complementary source of contrast, and k is an arbitrary scaling constant.

### Claim 22 (depends on 2)

The apparatus of claim 2 wherein the composite representation of the tissue specimen comprises, for each image from the sequence of images acquired by the primary imaging system:
a representation of the image from the sequence of images; and
a representation of the auxiliary image with a subregion of the auxiliary image indicating the location on the specimen holder where the image from the sequence of images was acquired, wherein the subregion of the auxiliary image is computed using one or more specimen holder positions from the one or more position sensors.

### Claim 23 (depends on 1)

A kit, comprising:
the apparatus of claim 1; and
a primary fluorescent nuclear contrast agents absorbing light emitted by the primary imaging system, and wherein a fluorescent emission wavelength of the primary fluorescent nuclear contrast agent corresponds to at least one of the spectrally separated channels.

### Claim 24 (depends on 23)

The kit of claim 23, further comprising:
a secondary fluorescent contrast agent having a fluorescent emission wavelength different from the fluorescent emission wavelength of the primary fluorescent nuclear contrast agent, wherein the secondary fluorescent contrast agent has specificity for a structure selected from cytoplasm, stroma, collagen, or muscle.

### Claim 25 (depends on 24)

The kit of claim 24, wherein the secondary fluorescent contrast agent is eosin-Y.

### Claim 26 (depends on 24)

The kit of claim 24, further including a fluid medium in which at least one of the primary fluorescent nuclear contrast agent or the secondary fluorescent contrast agent is soluble.

### Claim 27 (depends on 23)

The kit of claim 23, wherein the apparatus further comprises an identifier reader and wherein the kit further comprises an identifier.

### Claim 28

A method of real-time optical imaging of a tissue specimen comprising the steps of:
applying one or more fluorescent contrast agents to a tissue specimen, wherein at least one of the one or more fluorescent contrast agents is a nuclear contrast agent;
providing an apparatus, comprising:
a primary imaging system configured to use an illumination source to acquire images of a tissue specimen through one or more spectrally separated channels, at least one of the one or more spectrally separated channels is configured to detect a range of wavelengths distinct from the wavelength of the illumination source, the primary imaging system being an inverted microscope having a frame acquisition rate and configured to perform optical depth sectioning, the primary image system being configured to acquire a sequence of images;
an auxiliary imaging system, wherein the auxiliary imaging system is configured to acquire an auxiliary image of the tissue specimen, wherein the area of the auxiliary image is greater than the area of each image of the sequence of images acquired by the primary imaging system;
a specimen holder having a transparent window therewithin, the specimen holder disposed in a specimen plane at a focal plane of the primary imaging system, the specimen holder being configured to hold the tissue specimen on the transparent window, the specimen holder comprising one or more position sensors, wherein the specimen holder is configured to be translatable in the specimen plane, the one or more position sensors being configured to measure a specimen holder position, and wherein the specimen holder is configured to be translatable to the focal plane of the auxiliary imaging system;
wherein the primary imaging system and the auxiliary imaging system are both disposed on the same side of the specimen plane, the apparatus further comprising:
a user input device configured to accept user input, wherein the specimen holder is configured to translate in response to the user input in real-time;
a processing unit in electrical communication with the primary imaging system, the auxiliary imaging system, and the position sensors, wherein the processing unit is configured to execute a sequence of instructions on the sequence of images acquired by the primary imaging system, the auxiliary image, and at least one specimen holder position to form a representation of cell nuclei and to generate a composite representation of the tissue specimen; and
a display device in electrical communication with the processing unit, the display device being configured to display the composite representation of the tissue specimen in real-time,
situating the tissue specimen in the specimen holder;
positioning the specimen holder at the focal plane of the auxiliary imaging system;
causing the auxiliary imaging system to acquire the auxiliary image;
positioning the specimen holder at the focal plane of the primary imaging system;
causing the primary imaging system to acquire the sequence of images;
causing a processing unit to detect cell nuclei within the tissue specimen and to generate a composite representation of the tissue specimen;
causing the display device to display the composite representation of the tissue specimen in real-time; and
causing the specimen holder to translate in the specimen plane using the user input device.

### Claim 29 (depends on 28)

The method of claim 28, wherein acquiring the sequence of images comprises using multiphoton, confocal, light sheet, ultraviolet surface excitation, or structured illumination microscopy.

### Claim 30 (depends on 28)

The method of claim 28, wherein the primary imaging system is configured to adjust magnification by varying the area scanned, the method further including adjusting magnification of the primary imaging system.

### Claim 31 (depends on 28)

The method of claim 28, wherein:
each image of the sequence of images acquired by the primary imaging system is acquired through two or more spectrally separated channels;
and wherein at least one of the two or more spectrally separated channels is detecting light emitted by the nuclear contrast agent, and wherein at least one of the two or more spectrally separated channels is detecting light emitted from a complementary source of contrast.

### Claim 32 (depends on 28)

The method of claim 28, wherein the nuclear contrast agent is a rapid diffusion agent.

### Claim 33 (depends on 28)

The method of claim 28, wherein the nuclear contrast agent is an agent having enhancement of fluorescent emission by associating with cell nuclei.

### Claim 34 (depends on 28)

The method of claim 28, wherein the specimen holder includes a removable window, the method further comprising the steps of:
removing the transparent window from the specimen holder;
situating one or more tissue specimens onto the transparent window; and
inserting the transparent window back into the specimen holder.

### Claim 35 (depends on 28)

The method of claim 28, wherein the specimen holder includes a specimen divider configured to separate two or more tissue specimens.

### Claim 36 (depends on 28)

The method of claim 28, wherein the specimen holder includes a lid, the method further including covering the tissue specimen with the lid.

### Claim 37 (depends on 40)

The method of claim 40, wherein the lid includes a specimen guide, the method further including compressing the tissue specimen against the transparent window using the specimen guide.

### Claim 38 (depends on 28)

The method of claim 28, further including identifying surgical inks, sutures, or exogenous markings on the tissue specimen created during a medical procedure using the auxiliary image.

### Claim 39 (depends on 38)

The method of claim 38, further including measuring the distance between a selected location in any one of the sequence of images acquired by the primary imaging system and the surgical inks, sutures or exogenous markings on the tissue specimen using the representation of the auxiliary image.

### Claim 40 (depends on 38)

The method of claim 38, further including illuminating the tissue specimen with a narrow band illumination and identifying surgical inks, sutures, or exogenous markings from known absorption spectra.

### Claim 41 (depends on 38)

The method of claim 38, further including trans-illuminating the tissue specimen with the illumination source and identifying surgical inks, sutures, or exogenous markings.

### Claim 42 (depends on 28)

The method of claim 28, further including causing the processing unit to record at least one of: the sequence of images acquired by the primary imaging system, the specimen holder position, the auxiliary image, or the composite representation.

### Claim 43 (depends on 28)

The method of claim 28, further including:
causing the graphics processing unit to execute computer executable instructions for a series of parallel processing operations on the sequence of images.

### Claim 44 (depends on 43)

The method of claim 43, wherein the graphics processing unit includes computer executable instructions for a series of parallel processing operations for virtual transillumination microscopy processing.

### Claim 45 (depends on 28)

The method of claim 28, further including causing the processing unit to generate the composite representation of the tissue specimen, said composite representation comprising, for each image from the sequence of images acquired by the primary imaging system:
a representation of the image from the sequence of images; and
a representation of the auxiliary image with a subregion of the auxiliary image indicating the location on the specimen holder where the image from the sequence of images was acquired, wherein the subregion of the auxiliary image is computed using one or more specimen holder positions.

### Claim 46

A histological specimen cassette apparatus comprising:
an open-end receptacle, including a bottom wall and a specimen retaining structure;
a specimen cover, configured to close the open-end receptacle;
a specimen cover connector, configured to connect the open-end receptacle and the specimen cover;
a transparent window;
a transparent window connector, configured to connect the bottom wall or the specimen cover and the transparent window;
a plurality of perforations in one or more of the open-end receptacle, the transparent window or the specimen cover.

### Claim 47 (depends on 46)

The apparatus of claim 46, wherein the specimen cover connector is one or more of a notch, a tab, a hinge, or a magnet.

### Claim 48 (depends on 46)

The apparatus of claim 46, wherein the specimen retaining structure comprises a slanted wall with exterior surface.

### Claim 49 (depends on 46)

The apparatus of claim 46, wherein the transparent window comprises glass, fused quartz, silica, calcium fluoride or transparent plastic.

### Claim 50 (depends on 2)

The apparatus of claim 2, wherein the primary imaging system is configured to perform subsurface imaging.

### Claim 51 (depends on 50)

The apparatus of claim 50 wherein the primary imaging system comprises an illumination source configured to emit light having a red or an infrared wavelength.

### Claim 52 (depends on 51)

The apparatus of claim 51, wherein the illumination source is configured to emit light having a wavelength from 600 nm to 2000 nm.

### Claim 53 (depends on 2)

The apparatus of claim 2, wherein each image in the sequence of images acquired by the primary imaging system is acquired in response to a measurement by the one or more position sensors, and wherein the processing unit is configured to generate an averaged or a non-averaged image of the images in the sequence of images.

### Claim 54 (depends on 1)

The apparatus of claim 1, wherein each image in the sequence of images acquired by the primary imaging system is acquired through one or more spectrally separated channels; and
wherein at least one of the one or more spectrally separated channels is configured to detect light emitted by one or more sources selected from a surgical ink, a suture, and an exogenous marking.

### Claim 55 (depends on 54)

The apparatus of claim 54, wherein:
at least one of the two or more spectrally separated channels is configured to detect light emitted by a nuclear contrast agent; and
at least one of the two or more spectrally separated channels is configured to detect light emitted by one or more sources selected from a surgical ink, a suture, and an exogenous marking.

### Claim 56 (depends on 12)

The apparatus of claim 12, wherein the auxiliary imaging system is configured to be spatially separated from the primary imaging system.

### Claim 57 (depends on 23)

The kit of claim 23, further comprising:
one or more fluorescent signal source selected from a surgical ink, a suture, or an exogenous marker, the fluorescent signal source having a fluorescence emission signal different from that of the primary fluorescent contrast agent.

### Claim 58 (depends on 57)

The kit of claim 57, wherein the fluorescent signal source emits at a wavelength different from the fluorescent emission wavelength of the primary fluorescent contrast agent.

### Claim 59 (depends on 57)

The kit of claim 57, wherein the fluorescent signal source emits at a range of wavelengths that overlaps the fluorescent emission wavelength of the primary fluorescent contrast agent.

### Claim 60 (depends on 57)

The kit of claim 57, wherein the fluorescent signal source includes one or more of fluorescent microspheres, microcrystals, fluorescent dyes, or fluorescent plastics.

### Claim 61 (depends on 28)

The method of claim 28, wherein the composite representation comprises, for at least one image in the sequence of images acquired by the primary imaging system:
a representation of the image indicating two or more fiducial marks;
a representation of the auxiliary image; and
a representation of a distance between the two or more fiducial marks.

### Claim 62 (depends on 28)

The method of claim 28, wherein the processing unit is configured to generate a first and a second composite representations, wherein:
the first composite representation comprises:
a representation of a first image from the sequence of images acquired by the primary imaging system, the representation of the first image indicating a first fiducial mark, recorded in response to user input; and
a representation of the auxiliary image;
the second composite representation comprises:
a representation of a second image from the sequence of images acquired by the primary imaging system, the representation of the second image indicating a second fiducial mark, recorded in response to user input; and
a representation of the auxiliary image; and
a representation of a distance between the first and the second fiducial marks.
