# US 12,111,455 B2 · Detecting movements of a sample with respect to an objective > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,111,455 B2 - Document ID: US12111455B2 - Status: Granted - Kind: B2 - Application number: 17/482,752 - Filed: 2021-09-23 - Published: 2022-01-13 - Granted: 2024-10-08 - Earliest claimed: 2019-04-03 - Assignees: Abberior Instruments GmbH - Applicants: Abberior Instruments GmbH - Inventors: Roman Schmidt; Lars Kastrup - Web page: https://patentgrep.com/patent/US12111455B2 ## CPC - G02B 21/008: {Details of detection or image processing, including general computer control} - G02B 21/367: {providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison} - G02B 5/3083: {Birefringent or phase retarding elements (G02B5/3008, G02B5/3016 take precedence; systems for polarisation control G02B27/286; manufacturing phase modulating patterns by lithographic processes G03F7/001)} - G02B 21/0032: {Optical details of illumination, e.g. light-sources, pinholes, beam splitters, slits, fibers (G02B21/0036 - G02B21/008; means for illumination of specimens in general G02B21/06)} - G02B 21/02: Objectives - G02B 21/244: {using image analysis techniques} - G02B 21/245: {using auxiliary sources, detectors} - G02B 21/365: {Control or image processing arrangements for digital or video microscopes (G02B21/361, G02B21/362 take precedence)} - G02B 27/283: {used for beam splitting or combining} - G01N 21/6458: {Fluorescence microscopy (fluorescence microscopes per se G02B21/0076 and G02B21/16)} - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - G01N: INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES (measuring or testing apparatus or processes other than immunoassay, involving enzymes or microorganisms C12M, C12Q) ## Abstract An apparatus for detecting movements of a sample with respect to an objective includes imaging optics which include the objective, which have an image plane and which are configured to image light from at least one reference object that is connected to the sample arranged in front of the objective into reference object images in the image plane. The apparatus further includes a camera which is arranged in the image plane of the imaging optics and which is configured to record the reference object images at consecutive points in time, and an optical device arranged between the objective and the camera in a plane that is Fourier-conjugated with respect to the image plane. The optical device is configured to mask out low spatial frequencies of reference object images which the imaging optics image into the image plane. ## Claims ### Claim 1 An apparatus for detecting movements of a sample with respect to an objective, the apparatus comprising imaging optics which include the objective, have an image plane and are configured to image light from at least one reference object that is connected to the sample arranged in front of the objective into reference object images in the image plane, a camera which is arranged in the image plane of the imaging optics and which is configured to record the reference object images at consecutive points in time, and an optical device arranged between the objective and the camera, wherein the optical device is arranged in a plane that is Fourier-conjugated with respect to the image plane and wherein the optical device is configured to mask out low spatial frequencies of reference object images which the imaging optics image into the image plane, and wherein the illumination light source is configured and arranged to focus the illumination light on an optical axis of the objective into a back focal plane of the objective to wide-field illuminate the sample arranged in front of the objective with the illumination light. ### Claim 2 (depends on 1) The apparatus of claim 1, wherein the optical device is configured to mask out parts of light, which originate from a central area of a pupil of the objective, in front of the camera. ### Claim 3 (depends on 2) The apparatus of claim 2, wherein the imaging optics are configured to image the pupil of the objective into the plane that is Fourier-conjugated with respect to the image plane. ### Claim 4 (depends on 2) The apparatus of claim 2, wherein the optical device is selected from a circular disc, a mirror deflecting the low spatial frequencies to a further detector of the apparatus, a spatial light modulator (SLM) and a wire extending through a center of an image of the pupil of the objective in the plane that is Fourier-conjugated with respect to the image plane. ### Claim 5 (depends on 1) The apparatus of claim 1, comprising a band pass filter is arranged between the objective and the camera. ### Claim 6 (depends on 1) The apparatus of claim 1, comprising an illumination light source providing illumination light of one wavelength, wherein the illumination light source is configured and arranged for illuminating the reference objects that are connected to a sample arranged in front of the objective with the illumination light of the one wavelength. ### Claim 7 (depends on 6) The apparatus of claim 6, comprising an excitation light source providing excitation light that induces measuring light from the sample, wherein the one wavelength of the illumination light differs by at least 50 nm from wavelengths of the measuring light and of the excitation light. ### Claim 8 (depends on 6) The apparatus of claim 6, wherein the one wavelength is selected from an infrared range from 800 nm to 1.300 nm. ### Claim 9 (depends on 1) The apparatus of claim 1, wherein the objective is a multi-lens objective having a lens surface essentially coinciding with the back focal plane on the optical axis of the objective. ### Claim 10 (depends on 1) The apparatus of claim 1, comprising an illumination light source providing illumination light and a polarizing beam splitter which is arranged between the Fourier-conjugated plane and the objective, wherein the illumination light source is configured and arranged to couple-in the illumination light via the polarizing beam splitter, wherein the polarizing beam splitter is configured and arranged to transmit light that gets back into the objective after reflection of the illumination light at the at least one reference object that is connected to the sample arranged in front of the objective towards the camera due to its polarization direction. ### Claim 11 (depends on 10) The apparatus of claim 10, comprising a λ/4 waveplate arranged between the polarizing beam splitter and the objective. ### Claim 12 (depends on 1) The apparatus of claim 1, wherein a beam path between the objective and the camera starts at the objective as an infinity beam path. ### Claim 13 (depends on 1) The apparatus of claim 1, wherein the imaging optics comprise an optical element which introduces an imaging error that is configured to remove a symmetry of a deformation of images of the at least one reference object with increasing distance to a focal plane of the objective. ### Claim 14 (depends on 1) The apparatus of claim 1, wherein a focal length of the imaging optics is adjustable such that a reference object plane that is imaged by means of the imaging optics into the image plane is shiftable in the direction of an optical axis of the objective with respect to an object plane that is imaged by means of the objective. ### Claim 15 (depends on 1) The apparatus of claim 1, comprising a compensation device, wherein the compensation device is configured to compare the recorded reference object images to one another to gather any movements of the sample with respect to the objective. ### Claim 16 (depends on 15) The apparatus of claim 15, wherein the compensation device is configured to gather a lateral movement of the sample with respect to the object orthogonal to an optical axis of the objective from a shift of images of the at least one reference object between the recorded reference object images. ### Claim 17 (depends on 15) The apparatus of claim 15, wherein the compensation device is configured to gather a movement of the sample with respect to the objective along the optical axis of the objective from deformations of images of the at least one reference object between the recorded reference object images. ### Claim 18 (depends on 15) The apparatus of claim 15, wherein the compensation device is configured to gather a movement of the sample with respect to the objective along an optical axis of the objective from a similarity of images of the at least one reference object in the recorded reference object images with images of the reference objects or other reference objects in reference object comparison images. ### Claim 19 (depends on 15) The apparatus of claim 15, wherein the compensation device is configured to compensate any movements gathered by controlling a sample positioner for opposite movements of the sample with respect to the objective, or by controlling an adaptive optic causing a virtual movement of the sample with respect to the objective, or by calculating a correction for a microscopic image of the sample. ### Claim 20 (depends on 1) The apparatus of claim 1, wherein the imaging optics are configured to image further light from at least one optical boundary surface located between the sample and the objective or from at least one further reference object which is arranged at the objective into the reference object images in the image plane, and that the camera is configured to record the further light in the reference object images. ### Claim 21 (depends on 1) The apparatus of claim 1, wherein the imaging optics are configured to image the at least one reference object a reference object plane which is offset in the direction of the optical axis of the objective with respect to an object plane that is imaged by means of the objective. ### Claim 22 An apparatus for detecting movements of a sample with respect to an objective, the apparatus comprising an illumination device providing illumination light and a waveguide oriented transversely with respect to an optical axis of the objective, the illumination device being configured and arranged to couple the illumination light into a waveguide, imaging optics which include the objective, have an image plane and are configured to image light from at least one reference object that is connected to the sample arranged in front of the objective into reference object images in the image plane, a camera which is arranged in the image plane of the imaging optics and which is configured to record the reference object images at consecutive points in time, and an optical device arranged between the objective and the camera, wherein the optical device is arranged in a plane that is Fourier-conjugated with respect to the image plane and wherein the optical device is configured to mask out low spatial frequencies of reference object images which the imaging optics image into the image plane. ### Claim 23 (depends on 22) The apparatus of claim 22, wherein the waveguide is a plate of optically transparent material which serving as an object slide or coverslip of the sample. ### Claim 24 (depends on 22) The apparatus of claim 22, comprising a band pass filter is arranged between the objective and the camera. ### Claim 25 (depends on 22) The apparatus of claim 22, comprising an excitation light source providing excitation light that induces measuring light from the sample, wherein the one wavelength of the illumination light differs by at least 50 nm from wavelengths of the measuring light and of the excitation light. ### Claim 26 (depends on 22) The apparatus of claim 22, wherein the imaging optics comprise an optical element which introduces an imaging error that is configured to remove a symmetry of a deformation of images of the at least one reference object with increasing distance to a focal plane of the objective. ### Claim 27 (depends on 22) The apparatus of claim 22, wherein a focal length of the imaging optics is adjustable such that a reference object plane that is imaged by means of the imaging optics into the image plane is shiftable in the direction of an optical axis of the objective with respect to an object plane that is imaged by means of the objective. ### Claim 28 (depends on 22) The apparatus of claim 22, comprising a compensation device, wherein the compensation device is configured to compare the recorded reference object images to one another to gather any movements of the sample with respect to the objective. ### Claim 29 (depends on 28) The apparatus of claim 28, wherein the compensation device is configured to gather a lateral movement of the sample with respect to the object orthogonal to an optical axis of the objective from a shift of images of the at least one reference object between the recorded reference object images. ### Claim 30 (depends on 28) The apparatus of claim 28, wherein the compensation device is configured to gather a movement of the sample with respect to the objective along the optical axis of the objective from deformations of images of the at least one reference object between the recorded reference object images. ### Claim 31 (depends on 28) The apparatus of claim 28, wherein the compensation device is configured to gather a movement of the sample with respect to the objective along an optical axis of the objective from a similarity of images of the at least one reference object in the recorded reference object images with images of the reference objects or other reference objects in reference object comparison images. ### Claim 32 (depends on 28) The apparatus of claim 28, wherein the compensation device is configured to compensate any movements gathered by controlling a sample positioner for opposite movements of the sample with respect to the objective, or by controlling an adaptive optic causing a virtual movement of the sample with respect to the objective, or by calculating a correction for a microscopic image of the sample. ### Claim 33 (depends on 22) The apparatus of claim 22, wherein the imaging optics are configured to image further light from at least one optical boundary surface located between the sample and the objective or from at least one further reference object which is arranged at the objective into the reference object images in the image plane, and that the camera is configured to record the further light in the reference object images. ### Claim 34 (depends on 22) The apparatus of claim 22, wherein the imaging optics are configured to image the at least one reference object a reference object plane which is offset in the direction of the optical axis of the objective with respect to an object plane that is imaged by means of the objective. ### Claim 35 A method of detecting movements of a sample with respect to an objective, the method comprising imaging light from at least one reference object connected to the sample at consecutive points in time by means of the objective into reference object images in an image plane, selecting the at least one reference object such that it has an edge, in a plane that is Fourier-conjugated with respect to the image plane, masking out low spatial frequencies out of the reference object images in front of the camera in such a way that a diffraction pattern is formed at the edge in the reference object images, the diffraction pattern comprising a zero order intensity maximum which is reduced in its intensity due to the masked out low spatial frequencies, a first order intensity minimum and a first order intensity maximum following to the first order intensity minimum with intensity increasing regions neighboring the first order intensity minimum towards to the zeroth and first order intensity maxima, recording the reference object images in the image plane by a camera, and comparing the recorded reference object images to one another. ### Claim 36 (depends on 35) The method of claim 35 comprising selecting the at least one reference object such that the intensity of the light from the reference object drops across the edge by at least 90%, and a width of the edge in parallel to the image plane is smaller than the wavelength of the light. ### Claim 37 (depends on 36) The method of claim 36, comprising selecting the at least one reference object from a group consisting of a point-shaped marker, a bead and a nanorod having a diameter below the wavelength of the light. ### Claim 38 (depends on 35) The method of claim 35, wherein the step of comparing comprises gathering any movements of the sample with respect to the objective. ### Claim 39 (depends on 35) The method of claim 35, wherein the step of comparing comprises gathering a lateral movement of the sample with respect to the object orthogonal to an optical axis of the objective from a shift of images of the at least one reference object between the recorded reference object images. ### Claim 40 (depends on 35) The method of claim 35, wherein the step of comparing comprises gathering a movement of the sample with respect to the objective along the optical axis of the objective from deformations of images of the at least one reference object between the recorded reference object images. ### Claim 41 (depends on 38) The method of claim 38, further comprising compensating any movements of the sample with respect to the objective gathered by at least one of controlling a sample positioner for opposite movements of the sample with respect to the objective, controlling an adaptive optic causing a virtual movement of the sample with respect to the objective, and calculating a correction for a microscopic image of the sample. ### Claim 42 An apparatus for detecting movements of a sample with respect to an objective, the apparatus comprising imaging optics which include the objective, have an image plane and are configured to image light from at least one reference object that is connected to the sample arranged in front of the objective into reference object images in the image plane, a camera which is arranged in the image plane of the imaging optics and which is configured to record the reference object images at consecutive points in time, an optical device arranged between the objective and the camera, wherein the optical device is arranged in a plane that is Fourier-conjugated with respect to the image plane and wherein the optical device is configured to mask out low spatial frequencies of reference object images which the imaging optics image into the image plane, and a compensation device, wherein the compensation device is configured to compare the recorded reference object images to one another to gather any movements of the sample with respect to the objective. ### Claim 43 (depends on 42) A microscope comprising the apparatus of claim 42, wherein the microscope is a confocal microscope, a stimulated emission depletion (STED) microscope, a reversible saturable optical linear (Fluorescence) transitions (RESOLFT) microscope or a minimalemission fluxes (MINFLUX) microscope, and wherein the compensation device is configured to gather and compensate any movements of the sample with respect to the objective during operation of the microscope. ### Claim 44 (depends on 42) The apparatus of claim 42, wherein the compensation device is configured to gather a lateral movement of the sample with respect to the object orthogonal to an optical axis of the objective from a shift of images of the at least one reference object between the recorded reference object images. ### Claim 45 (depends on 42) The apparatus of claim 42, wherein the compensation device is configured to gather a movement of the sample with respect to the objective along the optical axis of the objective from deformations of images of the at least one reference object between the recorded reference object images. ### Claim 46 (depends on 42) The apparatus of claim 42, wherein the compensation device is configured to gather a movement of the sample with respect to the objective along an optical axis of the objective from a similarity of images of the at least one reference object in the recorded reference object images with images of the reference objects or other reference objects in reference object comparison images. ### Claim 47 (depends on 42) The apparatus of claim 42, wherein the compensation device is configured to compensate any movements gathered by controlling a sample positioner for opposite movements of the sample with respect to the objective, or by controlling an adaptive optic causing a virtual movement of the sample with respect to the objective, or by calculating a correction for a microscopic image of the sample. ### Claim 48 (depends on 42) The apparatus of claim 42, wherein the imaging optics are configured to image further light from at least one optical boundary surface located between the sample and the objective or from at least one further reference object which is arranged at the objective into the reference object images in the image plane, and that the camera is configured to record the further light in the reference object images. ### Claim 49 (depends on 42) The apparatus of claim 42, wherein the imaging optics are configured to image the at least one reference object a reference object plane which is offset in the direction of the optical axis of the objective with respect to an object plane that is imaged by means of the objective. ### Claim 50 (depends on 42) The apparatus of claim 42, comprising a band pass filter is arranged between the objective and the camera. ### Claim 51 (depends on 42) The apparatus of claim 42, comprising an illumination light source providing illumination light of one wavelength, wherein the illumination light source is configured and arranged for illuminating the reference objects that are connected to a sample arranged in front of the objective with the illumination light of the one wavelength, and an excitation light source providing excitation light that induces measuring light from the sample, wherein the one wavelength of the illumination light differs by at least 50 nm from wavelengths of the measuring light and of the excitation light. ### Claim 52 (depends on 42) The apparatus of claim 42, wherein the imaging optics comprise an optical element which introduces an imaging error that is configured to remove a symmetry of a deformation of images of the at least one reference object with increasing distance to a focal plane of the objective. ### Claim 53 (depends on 42) The apparatus of claim 42, wherein a focal length of the imaging optics is adjustable such that a reference object plane that is imaged by means of the imaging optics into the image plane is shiftable in the direction of an optical axis of the objective with respect to an object plane that is imaged by means of the object.