# US 2013/0155203 A1 · Image processing system and microscope system including the same

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2013/0155203 A1
- Document ID: US20130155203A1
- Status: Published application
- Kind: A1
- Application number: 13/719,988
- Filed: 2012-12-19
- Published: 2013-06-20
- Earliest claimed: 2011-12-20
- Assignees: Olympus Corporation
- Inventors: Nobuyuki WATANABE
- Web page: https://patentgrep.com/patent/US20130155203A1

## CPC

- H04N 13/207: using a single two-dimensional [2D] image sensor
- G06T 7/571: from focus
- G02B 21/365: {Control or image processing arrangements for digital or video microscopes (G02B21/361, G02B21/362 take precedence)}
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10056: Microscopic image
- G06T 2207/10148: Varying focus
- G06T 2207/20052: Discrete cosine transform [DCT]
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS

## Abstract

An image processing system includes an acquisition unit, a candidate value estimation unit, a cutoff frequency acquisition unit and a candidate value modification unit. The acquisition unit is configured to acquire an image of a sample taken via an optical system. The candidate value estimation unit is configured to estimate a candidate value of a 3D shape of the sample based on the image. The cutoff frequency acquisition unit is configured to acquire a cutoff frequency of the optical system based on information of the optical system. The candidate value modification unit is configured to perform at least one of data correction and data interpolation for the candidate value based on the cutoff frequency and calculate a modified candidate value.

## Claims

### Claim 1

An image processing system comprising:
an acquisition unit configured to acquire an image of a sample taken via an optical system;
a candidate value estimation unit configured to estimate a candidate value of a 3D shape of the sample based on the image;
a cutoff frequency acquisition unit configured to acquire a cutoff frequency of the optical system based on information of the optical system; and
a candidate value modification unit configured to perform at least one of data correction and data interpolation for the candidate value based on the cutoff frequency and calculate a modified candidate value.

### Claim 2 (depends on 1)

The image processing system according to claim 1, wherein
the acquisition unit acquires a plurality of images having focal positions different from each other for the single sample, and
the candidate value estimation unit estimates the candidate value of the 3D shape of the sample based on the plurality of images.

### Claim 3 (depends on 2)

The image processing system according to claim 2, further comprising:
a contrast evaluation unit configured to perform contrast evaluation for each pixel in each of the plurality of images and calculate a contrast evaluation value of each image of the plurality of images,
wherein the candidate value estimation unit selects a pixel in focus out of the pixels based on the contrast evaluation value and estimates the candidate value of the 3D shape of the sample.

### Claim 4 (depends on 2)

The image processing system according to claim 2, further comprising:
an all-in-focus image creation unit configured to create an all-in-focus image based on the modified candidate value and the plurality of images.

### Claim 5 (depends on 2)

The image processing system according to claim 2, further comprising:
a 3D reconstructed image creation unit configured to create a 3D reconstructed image based on the modified candidate value and the plurality of images.

### Claim 6 (depends on 1)

The image processing system according to claim 1, wherein
the candidate value modification unit includes a modified candidate value calculation unit configured to calculate the modified candidate value using correlation of a local region represented by the candidate value, and
an area of the local region is determined in accordance with the cutoff frequency.

### Claim 7 (depends on 6)

The image processing system according to claim 6, wherein
the correlation includes first correlation that is correlation between values of two points and second correlation that is correlation by a distance, and
a parameter for calculation of the first correlation and the second correlation is determined in accordance with the cutoff frequency.

### Claim 8 (depends on 7)

The image processing system according to claim 7, wherein
the modified candidate value calculation unit calculates the modified candidate value using a bilateral filter including the first correlation and the second correlation.

### Claim 9 (depends on 7)

The image processing system according to claim 7, wherein
the correlation is based on a probability density function.

### Claim 10 (depends on 6)

The image processing system according to claim 6, wherein
the data correction includes suppressing an isolated variation component included in the candidate value.

### Claim 11 (depends on 6)

The image processing system according to claim 6, wherein
the area is determined to be wider as the cutoff frequency becomes low.

### Claim 12 (depends on 1)

The image processing system according to claim 1, wherein
the candidate value modification unit performs noise reduction using correlation of a local region represented by the candidate value in the data correction.

### Claim 13 (depends on 12)

The image processing system according to claim 12, wherein
the candidate value modification unit suppresses a variation component determined as a small amplitude signal in the noise reduction.

### Claim 14 (depends on 1)

The image processing system according to claim 1, wherein
the data interpolation includes interpolating a value at a position inestimable by the candidate value estimation unit using a distance correlation of a local region.

### Claim 15 (depends on 14)

The image processing system according to claim 14, wherein
the distance correlation is determined to increase contribution of a wider region as the cutoff frequency becomes low.

### Claim 16 (depends on 1)

The image processing system according to claim 1, further comprising:
a 3D shape estimation unit configured to optimize the modified candidate value and estimate the 3D shape of the sample.

### Claim 17 (depends on 3)

The image processing system according to claim 3, further comprising:
a parameter determination unit configured to determine, based on the cutoff frequency, an evaluation parameter to be used for the contrast evaluation of the image,
wherein the contrast evaluation unit calculates the contrast evaluation value based on the evaluation parameter, and
the candidate value estimation unit includes an in-focus evaluation unit configured to evaluate an in-focus state of the pixel based on the contrast evaluation value.

### Claim 18 (depends on 17)

The image processing system according to claim 17, wherein
the parameter determination unit determines a band of a filter as the evaluation parameter based on the cutoff frequency, and
the contrast evaluation unit performs filtering processing using the filter having the band for the image, and calculates the contrast evaluation value based on a value obtained by the filtering processing.

### Claim 19 (depends on 18)

The image processing system according to claim 18, wherein
the parameter determination unit determines the band such that the band becomes high as the cutoff frequency becomes high.

### Claim 20 (depends on 17)

The image processing system according to claim 17, wherein
the contrast evaluation unit includes a plurality of filters having different bands,
the parameter determination unit determines a weighting coefficient for each of the plurality of filters as the evaluation parameter based on the cutoff frequency, and
the contrast evaluation unit performs filtering processing using the plurality of filters for the image, and calculates the contrast evaluation value by reflecting the weighting coefficient on a value obtained by the filtering processing.

### Claim 21 (depends on 20)

The image processing system according to claim 20, wherein
the parameter determination unit determines the weighting coefficient such that the weighting coefficient becomes high as the cutoff frequency becomes high.

### Claim 22 (depends on 17)

The image processing system according to claim 17, wherein
the parameter determination unit determines a cutoff coefficient as the evaluation parameter based on the cutoff frequency, and
the contrast evaluation unit acquires an orthonormal basis transformation coefficient by performing orthonormal basis transformation for the image, and calculates the contrast evaluation value by reflecting the cutoff coefficient on the orthonormal basis transformation coefficient.

### Claim 23 (depends on 22)

The image processing system according to claim 22, wherein the parameter determination unit determines the cutoff coefficient such that a weight of the orthonormal basis transformation coefficient of a high frequency becomes high as the cutoff frequency becomes high.

### Claim 24 (depends on 17)

The image processing system according to claim 17, further comprising a sample information acquisition unit configured to acquire optical characteristic information of the sample,
wherein the cutoff frequency acquisition unit uses the optical characteristic information when acquiring the cutoff frequency.

### Claim 25 (depends on 1)

A microscope system comprising the image processing system of claim 1,
wherein the optical system includes a variable magnification optical system.
