# US 2022/0284616 A1 · Microscopy System and Method for Processing a Microscope Image

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0284616 A1
- Document ID: US20220284616A1
- Status: Published application
- Kind: A1
- Application number: 17/683,133
- Filed: 2022-02-28
- Published: 2022-09-08
- Earliest claimed: 2021-03-02
- Assignees: Carl Zeiss Microscopy GmbH
- Applicants: Carl Zeiss Microscopy GmbH
- Inventors: Manuel Amthor; Daniel Haase
- Web page: https://patentgrep.com/patent/US20220284616A1

## CPC

- G06V 20/695: {Preprocessing, e.g. image segmentation}
- G06T 3/40: Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T 7/0012: {Biomedical image inspection}
- G06T 7/62: of area, perimeter, diameter or volume
- G06T 2207/10056: Microscopic image
- G06T 2207/20016: Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
- G06T 2207/20081: Training; Learning
- G06T 2207/30024: Cell structures in vitro; Tissue sections in vitro
- G06T 2207/30242: Counting objects in image
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

In a method for processing a microscope image, an input image is formed from a microscope image before being input into an image processing program. The image processing program comprises a learned model for image processing which is learned with training images that show structures with certain image properties. The image processing program calculates an image processing result from the input image. The microscope image is converted into the input image by means of an image conversion program in such a manner that image properties of structures in the input image are modified with respect to image properties of the structures of the microscope image so that they are closer to the image properties of the structures of the training images.

## Claims

### Claim 1

A method for processing a microscope image, comprising
receiving a microscope image;
inputting an input image formed from the microscope image into an image processing program comprising a learned model for image processing which is learned with training images that show structures with certain image properties;
calculating an image processing result from the input image by means of the image processing program; and
converting the microscope image into the input image by means of an image conversion program in such a manner that image properties of structures in the input image are modified with respect to image properties of the structures of the microscope image so that they are closer to the image properties of the structures of the training images.

### Claim 2 (depends on 1)

The method according to claim 1,
wherein image properties of the structures are geometry properties and/or brightness properties of the structures.

### Claim 3 (depends on 2)

The method according to claim 2,
wherein the geometry properties of the structures comprise a size, orientation or image distortion;
wherein the brightness properties relate to a brightness, a saturation or an image contrast of the structures.

### Claim 4 (depends on 1)

The method according to claim 1,
wherein the image conversion program determines the image properties of structures of the microscope image,
wherein target image properties are predetermined for image properties of structures of microscope images,
wherein the image conversion program changes the determined image properties to the target image properties, in order to calculate the input image from the microscope image.

### Claim 5 (depends on 1)

The method according to claim 1,
wherein the image conversion program determines a size of structures of the microscope image in pixels as an image property,
wherein a target pixel count is predetermined as a target image property,
wherein the image conversion program calculates a scaling factor by means of which the determined size of the structures is converted to the target pixel count,
wherein the image conversion program processes the microscope image with the scaling factor in order to generate the input image.

### Claim 6 (depends on 1)

The method according to claim 1,
wherein the image conversion program comprises a learned model for determining image properties that is trained using training images to determine image properties of structures of microscope images.

### Claim 7 (depends on 1)

The method according to claim 1,
wherein the image conversion program is a learned model for image conversion that is learned using images for which it is specified in the form of annotations how these images are to be converted in order to form input images.

### Claim 8 (depends on 7)

The method according to claim 7,
wherein the annotations respectively indicate a scaling factor.

### Claim 9 (depends on 1)

The method according to claim 1,
wherein the image conversion program is a learned model for image conversion, and
wherein the learned model for image conversion and the learned model for image processing are learned by means of a joint training with a joint optimization function.

### Claim 10 (depends on 1)

The method according to claim 1,
wherein the image conversion program, in order to determine a suitable conversion of the microscope image into the input image, tests a plurality of potential conversions, which are respectively applied to the microscope image in order to generate potential input images, wherein image properties of structures of the potential input images are subsequently evaluated according to a predetermined criterion and the potential input image with the best evaluation is selected as the input image.

### Claim 11 (depends on 1)

The method according to claim 1,
wherein the image conversion program takes into account contextual information regarding the microscope image when calculating the input image, the contextual information indicating one or more of the following:
microscope settings, a magnification of an employed objective, illumination properties or camera properties;
information regarding a measurement situation, an employed sample carrier type or an employed sample type.

### Claim 12 (depends on 1)

The method according to claim 1,
wherein the learned model for image processing calculates, for the input image, a classification, an image segmentation, a detection, an image enhancement, a reconstruction of image areas or an image-to-image mapping.

### Claim 13 (depends on 1)

The method according to claim 1,
wherein the image processing result is a result image and
wherein a conversion of the result image is performed that is inverse to the conversion by means of which the image conversion program generates the input image from the microscope image.

### Claim 14 (depends on 1)

A computer program with commands that, when executed by a computer, cause the execution of the method of claim 1.

### Claim 15

A microscopy system comprising
a microscope for capturing a microscope image; and
a computing device, which comprises an image processing program that is configured to process an input image formed from the microscope image into an image processing result, wherein the image processing program comprises a learned model for image processing which is learned with training images that show structures with certain image properties;
wherein the computing device comprises an image conversion program configured to convert the microscope image into the input image in such a manner that image properties of structures in the input image are modified with respect to image properties of the structures of the microscope image so that they are closer to the image properties of the structures of the training images.
