# US 2022/0036561 A1 · Method for image segmentation, method for training image segmentation model

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0036561 A1
- Document ID: US20220036561A1
- Status: Published application
- Kind: A1
- Application number: 17/497,954
- Filed: 2021-10-10
- Published: 2022-02-03
- Earliest claimed: 2020-07-30
- Assignees: Infervision Medical Technology Co., Ltd.
- Applicants: Infervision Medical Technology Co., Ltd.
- Inventors: Enyou LIU; Shaokang WANG; Kuan CHEN
- Web page: https://patentgrep.com/patent/US20220036561A1

## CPC

- G06T 7/11: Region-based segmentation
- G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques}
- G06F 18/253: {of extracted features}
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/09: Supervised learning
- G06T 5/70: Denoising; Smoothing
- G06T 7/187: involving region growing; involving region merging; involving connected component labelling
- G06T 7/194: involving foreground-background segmentation
- G06T 2207/10081: Computed x-ray tomography [CT]
- G06T 2207/10088: Magnetic resonance imaging [MRI]
- G06T 2207/10124: Digitally reconstructed radiograph [DRR]
- G06T 2207/20021: Dividing image into blocks, subimages or windows
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/20212: Image combination
- G06T 2207/20221: Image fusion; Image merging
- G06T 2207/30061: Lung
- G06T 2207/30101: Blood vessel; Artery; Vein; Vascular
- G06V 2201/03: Recognition of patterns in medical or anatomical images
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

The method for image segmentation includes: acquiring, according to an image to be segmented including a background, a mediastinum, an artery and a vein, a first segmentation result of the mediastinum, the artery, the vein and the background in a mediastinum region of the image to be segmented; acquiring, according to the image to be segmented, a second segmentation result of a blood vessel and the background in an epitaxial region of the image to be segmented; and acquiring, according to the first segmentation result and the second segmentation result, a segmentation result of the mediastinum, the artery, the vein and the background of the image to be segmented, so that the segmentation accuracy and the segmentation efficiency of the artery and the vein may be improved.

## Claims

### Claim 1

A method for image segmentation, comprising:
acquiring, according to an image to be segmented comprising a background, a mediastinum, an artery and a vein, a first segmentation result of the mediastinum, the artery, the vein and the background in a mediastinum region of the image to be segmented;
acquiring, according to the image to be segmented, a second segmentation result of a blood vessel and the background in an epitaxial region of the image to be segmented; and
acquiring, according to the first segmentation result and the second segmentation result, a segmentation result of the mediastinum, the artery, the vein and the background of the image to be segmented.

### Claim 2 (depends on 1)

The method according to claim 1, further comprising:
inputting the image to be segmented into a network model,
wherein the acquiring, according to an image to be segmented comprising a background, a mediastinum, an artery and a vein, a first segmentation result of the mediastinum, the artery, the vein and the background in a mediastinum region of the image to be segmented, comprises:
acquiring, according to the image to be segmented, the first segmentation result of the mediastinum, the artery, the vein and the background in the mediastinum region of the image to be segmented by the network model.

### Claim 3 (depends on 2)

The method according to claim 2, wherein the inputting the image to be segmented into a network model, comprises:
performing a dicing operation on the mediastinum region of the image to be segmented to obtain multiple dicing images, and each dicing image of the multiple dicing images comprising the mediastinum; and
inputting the multiple dicing images into the network model,
wherein the acquiring, according to the image to be segmented, the first segmentation result of the mediastinum, the artery, the vein and the background in the mediastinum region of the image to be segmented by the network model, comprises:
acquiring, according to the multiple dicing images, multiple sub-segmentation results of the mediastinum, the artery, the vein and the background in the mediastinum region corresponding to the multiple dicing images by the network model;
performing a combination operation on the multiple sub-segmentation results by a Gaussian smoothing process to obtain a combined segmentation result; and
performing a post-processing on the combined segmentation result by a connected region algorithm to obtain the first segmentation result.

### Claim 4 (depends on 3)

The method according to claim 3, wherein the performing a post-processing on the combined segmentation result by a connected region algorithm to obtain the first segmentation result, comprises:
acquiring a vein maximum connected region and an artery maximum connected region of the combined segmentation result by the connected region algorithm; and
removing, according to the vein maximum connected region and the artery maximum connected region, a noise point in the combined segmentation result to obtain the first segmentation result, and the noise point comprises a point that is both on the artery and the vein and a false positive point.

### Claim 5 (depends on 1)

The method according to claim 1, further comprising:
inputting the image to be segmented into a segmentation model,
wherein the acquiring, according to the image to be segmented, a second segmentation result of a blood vessel and the background in an epitaxial region of the image to be segmented, comprises:
acquiring, according to the image to be segmented, the second segmentation result of the blood vessel and the background in the epitaxial region by the segmentation model.

### Claim 6 (depends on 1)

The method according to claim 1, wherein the acquiring, according to the first segmentation result and the second segmentation result, a segmentation result of the mediastinum, the artery, the vein and the background of the image to be segmented, comprises:
acquiring, according to the artery and the vein in the first segmentation result and the blood vessel in the second segmentation result, the segmentation result of the mediastinum, the artery, the vein and the background of the image to be segmented by a region growth algorithm.

### Claim 7 (depends on 6)

The method according to claim 6, wherein the acquiring, according to the artery and the vein in the first segmentation result and the blood vessel in the second segmentation result, the segmentation result of the mediastinum, the artery, the vein and the background of the image to be segmented by a region growth algorithm, comprises:
performing, with the artery and the vein in the first segmentation result as a start point, a region growth on the artery and the vein in the first segmentation result along the blood vessel in the second segmentation result by a preset growth length, to obtain the segmentation result of the mediastinum, the artery, the vein and the background of the image to be segmented.

### Claim 8 (depends on 1)

The method according to claim 1, further comprising:
determining a sample image, and the sample image comprising a first label of a mediastinum, a background, an artery and a vein in a mediastinum region and a second label of a background and a blood vessel in an epitaxial region;
training a neural network based on the sample image to generate a network model for obtaining a first segmentation result of the mediastinum, the background, the artery and the vein in the mediastinum region, and the neural network being a 3D neural network; and
training a cascaded neural network based on the sample image to generate a segmentation model for obtaining a second segmentation result of the background and the blood vessel in the epitaxial region, and the cascaded neural network comprising a first neural network for feature extraction and a second neural network for generating the second segmentation result.

### Claim 9 (depends on 8)

The method according to claim 8, further comprising:
performing a dicing operation on the mediastinum region of the sample image to obtain multiple dicing images, wherein each dicing image of the multiple dicing images comprises the mediastinum, and
the training a neural network based on the sample image to generate a network model for obtaining a first segmentation result of the mediastinum, the background, the artery and the vein in the mediastinum region, comprises:
training the neural network based on the multiple dicing images to generate the network model.

### Claim 10 (depends on 8)

The method according to claim 8, wherein the training a cascaded neural network based on the sample image to generate a segmentation model for obtaining a second segmentation result of the background and the blood vessel in the epitaxial region, comprises:
performing a down-sampling operation on the sample image by the first neural network to obtain multiple first feature maps;
performing an up-sampling operation and a fusion operation on the multiple first feature maps by the second neural network to obtain a second feature map;
activating the second feature map by using a classifier to obtain the second segmentation result of the background and the blood vessel in the epitaxial region;
obtaining a loss function value of the cascaded neural network according to the second segmentation result and the second label; and
updating parameters of the cascaded neural network according to the loss function value,
the first neural network is a deep residual network, and the second neural network is a feature pyramid network.

### Claim 11 (depends on 10)

The method according to claim 10, further comprising:
performing a max pooling operation on a region where the blood vessel in the second label is located to obtain a target region of the sample image after the region where the blood vessel in the second label is located is expanded,
wherein the obtaining a loss function value of the cascaded neural network according to the second segmentation result and the second label, comprises:
obtaining the loss function value of the cascaded neural network according to the second segmentation result corresponding to the target region and the second label.

### Claim 12

An electronic apparatus, comprising:
a processor;
a memory configured to store executable instructions of the processor;
wherein the processor is configured to perform the following steps:
acquiring, according to an image to be segmented comprising a background, a mediastinum, an artery and a vein, a first segmentation result of the mediastinum, the artery, the vein and the background in a mediastinum region of the image to be segmented;
acquiring, according to the image to be segmented, a second segmentation result of a blood vessel and the background in an epitaxial region of the image to be segmented; and
acquiring, according to the first segmentation result and the second segmentation result, a segmentation result of the mediastinum, the artery, the vein and the background of the image to be segmented.

### Claim 13 (depends on 12)

The electronic apparatus according to claim 12, wherein the processor further performs the following step:
inputting the image to be segmented into a network model,
wherein the acquiring, according to an image to be segmented comprising a background, a mediastinum, an artery and a vein, a first segmentation result of the mediastinum, the artery, the vein and the background in a mediastinum region of the image to be segmented, comprises:
acquiring, according to the image to be segmented, the first segmentation result of the mediastinum, the artery, the vein and the background in the mediastinum region of the image to be segmented by the network model.

### Claim 14 (depends on 13)

The electronic apparatus according to claim 13, wherein the inputting the image to be segmented into a network model, comprises:
performing a dicing operation on the mediastinum region of the image to be segmented to obtain multiple dicing images, and each dicing image of the multiple dicing images comprising the mediastinum; and
inputting the multiple dicing images into the network model,
wherein the acquiring, according to the image to be segmented, the first segmentation result of the mediastinum, the artery, the vein and the background in the mediastinum region of the image to be segmented by the network model, comprises:
acquiring, according to the multiple dicing images, multiple sub-segmentation results of the mediastinum, the artery, the vein and the background in the mediastinum region corresponding to the multiple dicing images by the network model;
performing a combination operation on the multiple sub-segmentation results by a Gaussian smoothing process to obtain a combined segmentation result; and
performing a post-processing on the combined segmentation result by a connected region algorithm to obtain the first segmentation result.

### Claim 15 (depends on 12)

The electronic apparatus according to claim 12, wherein the processor further performs the following step:
inputting the image to be segmented into a segmentation model,
wherein the acquiring, according to the image to be segmented, a second segmentation result of a blood vessel and the background in an epitaxial region of the image to be segmented, comprises:
acquiring, according to the image to be segmented, the second segmentation result of the blood vessel and the background in the epitaxial region by the segmentation model.

### Claim 16 (depends on 12)

The electronic apparatus according to claim 12, wherein the acquiring, according to the first segmentation result and the second segmentation result, a segmentation result of the mediastinum, the artery, the vein and the background of the image to be segmented, comprises:
acquiring, according to the artery and the vein in the first segmentation result and the blood vessel in the second segmentation result, the segmentation result of the mediastinum, the artery, the vein and the background of the image to be segmented by a region growth algorithm.

### Claim 17 (depends on 12)

The electronic apparatus according to claim 12, wherein the processor further performs the following steps:
determining a sample image, and the sample image comprising a first label of a mediastinum, a background, an artery and a vein in a mediastinum region and a second label of a background and a blood vessel in an epitaxial region;
training a neural network based on the sample image to generate a network model for obtaining a first segmentation result of the mediastinum, the background, the artery and the vein in the mediastinum region, and the neural network being a 3D neural network; and
training a cascaded neural network based on the sample image to generate a segmentation model for obtaining a second segmentation result of the background and the blood vessel in the epitaxial region, and the cascaded neural network comprising a first neural network for feature extraction and a second neural network for generating the second segmentation result.

### Claim 18 (depends on 17)

The electronic apparatus according to claim 17, wherein the processor further performs the following step:
performing a dicing operation on the mediastinum region of the sample image to obtain multiple dicing images, wherein each dicing image of the multiple dicing images comprises the mediastinum,
wherein the training a neural network based on the sample image to generate a network model for obtaining a first segmentation result of the mediastinum, the background, the artery and the vein in the mediastinum region, comprises:
training the neural network based on the multiple dicing images to generate the network model.

### Claim 19 (depends on 17)

The electronic apparatus according to claim 17, wherein the training a cascaded neural network based on the sample image to generate a segmentation model for obtaining a second segmentation result of the background and the blood vessel in the epitaxial region, comprises:
performing a down-sampling operation on the sample image by the first neural network to obtain multiple first feature maps;
performing an up-sampling operation and a fusion operation on the multiple first feature maps by the second neural network to obtain a second feature map;
activating the second feature map by using a classifier to obtain the second segmentation result of the background and the blood vessel in the epitaxial region;
obtaining a loss function value of the cascaded neural network according to the second segmentation result and the second label; and
updating parameters of the cascaded neural network according to the loss function value,
wherein the first neural network is a deep residual network, and the second neural network is a feature pyramid network.

### Claim 20

A non-transitory computer readable storage medium having stored thereon executable instructions configured to cause a processor to perform the following steps:
acquiring, according to an image to be segmented comprising a background, a mediastinum, an artery and a vein, a first segmentation result of the mediastinum, the artery, the vein and the background in a mediastinum region of the image to be segmented;
acquiring, according to the image to be segmented, a second segmentation result of a blood vessel and the background in an epitaxial region of the image to be segmented; and
acquiring, according to the first segmentation result and the second segmentation result, a segmentation result of the mediastinum, the artery, the vein and the background of the image to be segmented.
