# US 2022/0012457 A1 · Image processing method, microscope, image processing system, and medium based on artificial intelligence

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0012457 A1
- Document ID: US20220012457A1
- Status: Published application
- Kind: A1
- Application number: 17/482,177
- Filed: 2021-09-22
- Published: 2022-01-13
- Earliest claimed: 2019-09-27
- Assignees: Tencent Technology (Shenzhen) Company Limited
- Applicants: Tencent Technology (Shenzhen) Company Limited
- Inventors: Baochang Han; Xiao Han; Yong Chen; Peng Yang
- Web page: https://patentgrep.com/patent/US20220012457A1

## CPC

- G02B 21/365: {Control or image processing arrangements for digital or video microscopes (G02B21/361, G02B21/362 take precedence)}
- G06V 20/695: {Preprocessing, e.g. image segmentation}
- G02B 21/34: Microscope slides, e.g. mounting specimens on microscope slides
- G02B 21/367: {providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison}
- G06T 7/11: Region-based segmentation
- G06T 7/62: of area, perimeter, diameter or volume
- G06V 20/69: Microscopic objects, e.g. biological cells or cellular parts
- G06T 2207/10056: Microscopic image
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30024: Cell structures in vitro; Tissue sections in vitro
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Embodiments of the present disclosure provide an image processing method based on artificial intelligence (AI) and an image processing system. The method includes: obtaining a feature recognition result of an image by performing image processing on the image to recognize a feature of the image and the image being obtained by performing image acquisition on a section of a patient using a digital slide scanner to generate a whole slide image (WSI) as the image; determining an imaging area of the section within a field of view of an eyepiece of a microscope with which real-time imaging is performed on the section; determining, within the image, an image area corresponding to the imaging area of the section and acquiring, from the feature recognition result of the image, a target feature recognition result of the image area; and superimposing the target feature recognition result on the imaging area of the section.

## Claims

### Claim 1

An image processing method performed by a computing device, the method comprising:
obtaining a feature recognition result of an image, the feature recognition result being obtained by performing image processing on the image to recognize a feature in the image, and the image being obtained by performing image acquisition on a section of a patient using a digital slide scanner to generate a whole slide image (WSI) as the image;
determining an imaging area of the section within a field of view of an eyepiece of a microscope with which real-time imaging is performed on the section;
determining, within the image, an image area corresponding to the imaging area of the section and acquiring, from the feature recognition result of the image, a target feature recognition result of the image area; and
superimposing the target feature recognition result on the imaging area of the section.

### Claim 2 (depends on 1)

The image processing method according to claim 1, wherein the feature recognition result includes an identifier of the feature and a boundary line of the feature.

### Claim 3 (depends on 2)

The image processing method according to claim 2, further comprising:
acquiring verification indication information of the section; and
performing imaging on the section using the microscope based on the verification indication information,
verification indication information of the image being generated as the verification indication information of the section corresponding to the image based on the identifier of the feature.

### Claim 4 (depends on 1)

The image processing method according to claim 1, wherein the determining, within the image, an image area corresponding to the imaging area of the section comprises:
determining a display range of the imaging area in a display of the microscope configured to display the target feature recognition result;
determining image coordinate values corresponding to a center point of the field of view of the eyepiece of the microscope in the image; and
determining the image area corresponding to the imaging area based on the display range and the image coordinate values.

### Claim 5 (depends on 4)

The image processing method according to claim 4, wherein the determining image coordinate values corresponding to a center point of the field of view of the eyepiece of the microscope in the image comprises:
determining section coordinate values of the center point of the field of view of the eyepiece in the section;
determining physical dimensions of the section and pixel dimensions of the image; and
determining the image coordinate values based on the physical dimensions, the pixel dimensions, and the section coordinate values of the center point of the field of view of the eyepiece,
(
x
2
,
y
2
)
=
(
x
1
*
W
2
W
1
,
y
1
*
H
2
H
1
)
the image coordinate values being denoted as (x2, y2), the section coordinate values of the center point of the field of view of the eyepiece being denoted as (x1, y1), physical dimensions of the section in lateral and longitudinal directions being respectively denoted as W1 and H1, and pixel dimensions of the image in lateral and longitudinal directions being respectively denoted as W2 and H2.

### Claim 6 (depends on 5)

The image processing method according to claim 5, wherein the determining section coordinate values of the center point of the field of view of the eyepiece in the section comprises:
determining first coordinate values of the center point of the field of view of the eyepiece in a coordinate system of an object stage of the microscope;
determining second coordinate values of an upper left corner of the section in the coordinate system of the object stage of the microscope; and
determining the section coordinate values of the center point of the field of view of the eyepiece in a section coordinate system based on the first coordinate values and the second coordinate values,
the section coordinate system using the upper left corner of the section as an origin, a horizontal direction of the section as a lateral axis, and a vertical direction of the section as a vertical axis, and the first coordinate values, the second coordinate values, and the section coordinate values indicating physical lengths.

### Claim 7 (depends on 4)

The image processing method according to claim 4, wherein the display range is indicated by a display radius; and
the determining the image area corresponding to the imaging area based on the display range and the image coordinate values comprises:
determining a recognition radius corresponding to the display radius in the image;
determining an area with the image coordinate values as a center and the recognition radius as a radius in the image as the image area; and
using a recognition result of the image area as the target feature recognition result.

### Claim 8 (depends on 7)

The image processing method according to claim 7, wherein the determining a recognition radius corresponding to the display radius in the image comprises:
determining an acquisition magnification of the image;
determining an objective magnification of the microscope; and
determining the recognition radius based on the display radius, the acquisition magnification, and the objective magnification,
r
2
=
r
1
*
m
1
m
2
r2 indicating the recognition radius, r1 indicating the display radius, m1 indicating the acquisition radius, and m2 indicating the objective magnification.

### Claim 9

An image processing system, comprising:
a digital slide scanner, configured to perform image acquisition on a section of a patient to generate a whole slide image (WSI) as an image;
a microscope, configured to perform real-time imaging on the section;
a processor; and
a memory,
the memory storing a plurality of instructions that, when executed by the processor, perform a plurality of operations including:
obtaining a feature recognition result of the image, the feature recognition result being obtained by performing image processing on the image to recognize a feature in the image;
determining an imaging area of the section within a field of view of an eyepiece of the microscope with which the real-time imaging is performed on the section;
determining, within the image, an image area corresponding to the imaging area of the section and acquiring, from the feature recognition result of the image, a target feature recognition result of the image area; and
superimposing the target feature recognition result on the imaging area of the section.

### Claim 10 (depends on 9)

The image processing system according to claim 9, wherein the feature recognition result includes an identifier of the feature and a boundary line of the feature.

### Claim 11 (depends on 10)

The image processing system according to claim 10, wherein the plurality of operations further comprise:
acquiring verification indication information of the section; and
performing imaging on the section using the microscope based on the verification indication information,
verification indication information of the image being generated as the verification indication information of the section corresponding to the image based on the identifier of the feature.

### Claim 12 (depends on 9)

The image processing system according to claim 9, wherein the determining, within the image, an image area corresponding to the imaging area of the section comprises:
determining a display range of the imaging area in a display of the microscope configured to display the target feature recognition result;
determining image coordinate values corresponding to a center point of the field of view of the eyepiece of the microscope in the image; and
determining the image area corresponding to the imaging area based on the display range and the image coordinate values.

### Claim 13 (depends on 12)

The image processing system according to claim 12, wherein the determining image coordinate values corresponding to a center point of the field of view of the eyepiece of the microscope in the image comprises:
determining section coordinate values of the center point of the field of view of the eyepiece in the section;
determining physical dimensions of the section and pixel dimensions of the image; and
determining the image coordinate values based on the physical dimensions, the pixel dimensions, and the section coordinate values of the center point of the field of view of the eyepiece,
(
x
2
,
y
2
)
=
(
x
1
*
W
2
W
1
,
y
1
*
H
2
H
1
)
the image coordinate values being denoted as (x2, y2), the section coordinate values of the center point of the field of view of the eyepiece being denoted as (x1, y1), physical dimensions of the section in lateral and longitudinal directions being respectively denoted as W1 and H1, and pixel dimensions of the image in lateral and longitudinal directions being respectively denoted as W2 and H2.

### Claim 14 (depends on 13)

The image processing system according to claim 13, wherein the determining section coordinate values of the center point of the field of view of the eyepiece in the section comprises:
determining first coordinate values of the center point of the field of view of the eyepiece in a coordinate system of an object stage of the microscope;
determining second coordinate values of an upper left corner of the section in the coordinate system of the object stage of the microscope; and
determining the section coordinate values of the center point of the field of view of the eyepiece in a section coordinate system based on the first coordinate values and the second coordinate values,
the section coordinate system using the upper left corner of the section as an origin, a horizontal direction of the section as a lateral axis, and a vertical direction of the section as a vertical axis, and the first coordinate values, the second coordinate values, and the section coordinate values indicating physical lengths.

### Claim 15 (depends on 12)

The image processing system according to claim 12, wherein the display range is indicated by a display radius; and
the determining the image area corresponding to the imaging area based on the display range and the image coordinate values comprises:
determining a recognition radius corresponding to the display radius in the image;
determining an area with the image coordinate values as a center and the recognition radius as a radius in the image as the image area; and
using a recognition result of the image area as the target feature recognition result.

### Claim 16 (depends on 15)

The image processing system according to claim 15, wherein the determining a recognition radius corresponding to the display radius in the image comprises:
determining an acquisition magnification of the image;
determining an objective magnification of the microscope; and
determining the recognition radius based on the display radius, the acquisition magnification, and the objective magnification,
r
2
=
r
1
*
m
1
m
2
r2 indicating the recognition radius, r1 indicating the display radius, m1 indicating the acquisition radius, and m2 indicating the objective magnification.

### Claim 17

A non-transitory computer-readable storage medium in connection with an image processing system including a digital slide scanner, configured to perform image acquisition on a section of a patient to generate a whole slide image (WSI) as an image, a microscope, configured to perform real-time imaging on the section, and a processor, the non-transitory computer-readable storage medium storing instructions, the instructions, when executed by the processor, causing the image processing system to perform a plurality of operations including:
obtaining a feature recognition result of the image, the feature recognition result being obtained by performing image processing on the image to recognize a feature in the image;
determining an imaging area of the section within a field of view of an eyepiece of the microscope with which the real-time imaging is performed on the section;
determining, within the image, an image area corresponding to the imaging area of the section and acquiring, from the feature recognition result of the image, a target feature recognition result of the image area; and
superimposing the target feature recognition result on the imaging area of the section.

### Claim 18 (depends on 17)

The non-transitory computer-readable storage medium according to claim 17, wherein the feature recognition result includes an identifier of the feature and a boundary line of the feature.

### Claim 19 (depends on 18)

The non-transitory computer-readable storage medium according to claim 18, wherein the plurality of operations further comprise:
acquiring verification indication information of the section; and
performing imaging on the section using the microscope based on the verification indication information,
verification indication information of the image being generated as the verification indication information of the section corresponding to the image based on the identifier of the feature.

### Claim 20 (depends on 17)

The non-transitory computer-readable storage medium according to claim 17, wherein the determining, within the image, an image area corresponding to the imaging area of the section comprises:
determining a display range of the imaging area in a display of the microscope configured to display the target feature recognition result;
determining image coordinate values corresponding to a center point of the field of view of the eyepiece of the microscope in the image; and
determining the image area corresponding to the imaging area based on the display range and the image coordinate values.
