# US 2015/0032414 A1 · Method for the 3-Dimensional Measurement of a Sample With a Measuring System Comprising a Laser Scanning Microscope and Such Measuring System

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2015/0032414 A1
- Document ID: US20150032414A1
- Status: Published application
- Kind: A1
- Application number: 14/368,936
- Filed: 2012-01-05
- Published: 2015-01-29
- Earliest claimed: 2011-12-28
- Assignees: Femtonics Kft.
- Inventors: Balázs Rozsa; Gergely Katona; Ferenc Csikor; Pál Maak
- Web page: https://patentgrep.com/patent/US20150032414A1

## CPC

- G01B 11/24: for measuring contours or curvatures
- G01B 9/04: Measuring microscopes
- G06F 3/01: Input arrangements or combined input and output arrangements for interaction between user and computer (G06F3/16 takes precedence)
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
- G01B: MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

The invention relates to a method for the 3-dimensional measurement of a sample with a measuring system having a 3-dimensional measuring space and comprising a laser scanning microscope, characterised by—providing the measuring system with a 3-dimensional virtual reality device,—creating the 3-dimensional virtual space of the measuring space using the 3-dimensional virtual reality device,—allowing for selecting an operation in the virtual space,—providing real-time unidirectional or bidirectional convection between the measuring space and the virtual space such that an operation selected in the virtual space is performed in the measuring space and data measured in the measuring space is displayed in the virtual space. The invention further relates to a measuring system for the 3-dimensional measurement of a sample, the measuring system having a 3-dimensional measuring space and comprising a laser scanning microscope, characterised by further comprising a 3-dimensional virtual reality device for displaying a 3-dimensional virtual space of the measuring space, and a real-time unidirectional or bidirectional connection is provided between the laser scanning microscope and the 3-dimensional virtual reality device.

## Claims

### Claim 1

Method for the 3-dimensional measurement of a sample with a measuring system having a 3-dimensional measuring space and comprising a laser scanning microscope, characterised by
providing the measuring system with a 3-dimensional virtual reality device,
creating, using the 3-dimensional virtual reality device, the 3-dimensional virtual space of the measuring space in a real space region that is spaced from the measuring space,
allowing for selecting an operation in the virtual space,
providing real-time bidirectional connection between the measuring space and the virtual space such that an operation selected in the virtual space is performed in the measuring space and data measured in the measuring space is displayed in the virtual space.

### Claim 2 (depends on 1)

The method according to claim 1, characterised by
scanning a sample or a portion of a sample placed in the measuring space of the laser scanning microscope in 3-dimension,
displaying the scanned sample in the virtual space,
allowing for selecting a virtual configuration comprising at least a point and/or curve and/or region,
calculating the measuring space coordinates corresponding to the virtual configuration,
based on the measuring space coordinates controlling the laser scanning microscope so as to scan the sample according to the virtual configuration and to perform measurement and/or interact with the sample in the course of scanning.

### Claim 3 (depends on 2)

The method according to claim 2, characterised by that the virtual configuration is a quasi continuous curve consisting of points in 3-dimension or a curve interpolated on the defining points and comprising the steps of: displacing the points of the curve or the defining points of the interpolated curve in the virtual space; and modifying the measuring coordinates in the measuring space in accordance with the displacement.

### Claim 4 (depends on 2)

The method according to claim 2, characterised by that the virtual space is a quasi continuous curve consisting of points in 3-dimension or a curve interpolated on the defining points and comprising the steps of: controlling functions and/or algorithms affecting the points of the curve or the defining points of the interpolated curve; displaying these; and modifying the measuring coordinates in the measuring space accordingly.

### Claim 5 (depends on 2)

The method according to claim 2, characterised by
scanning the sample in a plurality of planes that are orthogonal or transversal to the optical axis of the laser scanning microscope in the course of the 3-dimensional scanning of the sample, and
displaying the scanned planes in the virtual space.

### Claim 6 (depends on 5)

The method according to claim 5, characterised by displaying the scanned sample or portion thereof in the virtual space as spatial fog.

### Claim 7 (depends on 5)

The method according to claim 5, characterised by displaying the scanned sample or portion thereof in the virtual space as spatial surface element and using one or more simulated light sources to increase the 3-dimensional effect.

### Claim 8 (depends on 1)

The method according to claim 1, characterised by
providing the measuring system with an operation device serving to carry out physical operation, which operation device may coincide with the measuring laser beam,
selecting an operation in the virtual space for the operation device of the measuring space,
performing the operation selected in the virtual space with the operation device in the measuring space.

### Claim 9 (depends on 8)

The method according to claim 8, characterised by that the physical operation is a physical interaction or the measurement of physical parameters, and the operation device serving to carry out the physical operation is a robot surgical device, knife, laser ablation device, ultrasound coagulator, laser coagulator, micro injector, vacuum suction device, optical cable, prism, grid lens, electric stimulator, a measuring device to be inserted in the sample, preferably electrode, micro pipette for patch-clamp technique, extracellular recording electrode, endoscopic device, electrophoretic device.

### Claim 10 (depends on 1)

The method according to claim 1, characterised by providing the 3-dimensional virtual reality device with a 3-dimensional input device—preferably a 3-dimensional marker—for selecting operations.

### Claim 11 (depends on 8)

The method according to claim 8, characterised by providing the 3-dimensional virtual reality device with a 3-dimensional mouse or mechanical arm the coordinate system of which is synchronised with the virtual space and being suitable for drawing and selecting 3-dimensional spatial curves and points for selecting operations.

### Claim 12 (depends on 11)

The method according to claim 11, characterised by that the spatial position of the pointing device of the 3-dimensional input device provided for selecting operations coincides with the spatial position of a virtual marker belonging to the device or it is offset relative to it by a given spatial vector.

### Claim 13 (depends on 8)

The method according to claim 8, characterised by changing the optical contrast or visibility of the virtual reality in a given local virtual environment of the 3-dimensional input device and optionally its 3-dimensional virtual marker (actuated by the input device), thereby preferably increasing the contrast for better viewing an interaction.

### Claim 14 (depends on 1)

The method according to claim 1, characterised by providing a 3-dimensional display device—preferably an autostereoscopic display—within the 3-dimensional virtual reality device in order to display data measured in the measuring space.

### Claim 15 (depends on 1)

The method according to claim 1, characterised by providing the 3-dimensional virtual reality device with a 3-dimensional display device—preferably a stereoscopic display—and an active or passive stereoscopic user device—preferably glasses—in order to display data measured in the measuring space.

### Claim 16 (depends on 14)

The method according to claim 14, characterised by providing for head position tracking within the 3-dimensional virtual reality device and displaying the virtual space by the 3-dimensional display device in accordance with the position of the user's head.

### Claim 17 (depends on 1)

The method according to claim 1, characterised by
providing the measuring system with more than one 3-dimensional virtual reality devices,
creating more than one 3-dimensional virtual spaces of the measuring space by the more than one 3-dimensional virtual reality devices for more than one users.

### Claim 18 (depends on 17)

The method according to claim 17, characterised by providing real time connection over a local or global network between the more than one 3-dimensional virtual reality devices and the measuring system.

### Claim 19 (depends on 1)

The method according to claim 1, characterised by providing a common 3-dimensional virtual reality device for more than one measuring systems, and creating a common 3-dimensional virtual space of the more than one measuring spaces with the common 3-dimensional virtual reality device.

### Claim 20 (depends on 1)

The method according to claim 1, characterised by that the measuring system comprises a two-photon laser scanning microscope or a confocal laser scanning microscope.

### Claim 21

Measuring system for the 3-dimensional measurement of a sample, the measuring system having a 3-dimensional measuring space and comprising a laser scanning microscope, characterised by further comprising a 3-dimensional virtual reality device for displaying a 3-dimensional virtual space of the measuring space in a real space region that is spaced from the measuring space, and a real-time bidirectional connection is provided between the laser scanning microscope and the 3-dimensional virtual reality device.

### Claim 22 (depends on 21)

The measuring system according to claim 21, characterised by comprising an operation device for performing physical operation and a real-time unidirectional or bidirectional connection is provided between the operation device and the 3-dimensional virtual reality device.

### Claim 23 (depends on 1)

Measuring system for the 3-dimensional measurement of a sample, the measuring system having a 3-dimensional measuring space and comprising a laser scanning microscope and optionally other operation devices and being provided with a controlling system, characterised by comprising a 3-dimensional virtual reality device having real-time bidirectional connection with the controlling system for performing the method according to claim 1.

### Claim 24 (depends on 1)

The method according to claim 1, characterised by operating the measuring system at one or more wavelengths simultaneously or nearly simultaneously.

### Claim 25 (depends on 24)

The method according to claim 24, characterised by measuring at one or more wavelengths and simultaneously or nearly simultaneously performing physical operation at one or more different wavelengths.

### Claim 26 (depends on 20)

The method according to claim 20, characterised by providing z-focusing and/or xy-scanning by mechanical displacement in the 3-dimensional laser scanning microscope.

### Claim 27 (depends on 1)

The method and measuring system according to claim 1, characterised by displaying in the 3-dimensional virtual space the error between the physical displacement realised in the course of spatial scanning and the configuration selected for measuring, and modifying the spatial trajectory with the 3-dimensional input device so as to decrease the spatial error of the measurement.

### Claim 28 (depends on 20)

The method and measuring system according to claim 20, characterised by using a 3-dimensional laser scanning microscope having an acousto-optic deflector system or having a holographic scanner unit, or by using a 3-dimensional laser scanning microscope which performs 3-dimensional scanning by spatio-temporal multiplexing (i.e. by spatial impulse separation), or by using a 3-dimensional laser scanning microscope which performs 3-dimensional scanning by controlling mechanically displaceable lenses with electric field.
