# US 2020/0402204 A1 · Medical imaging using neural networks

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0402204 A1
- Document ID: US20200402204A1
- Status: Published application
- Kind: A1
- Application number: 16/904,981
- Filed: 2020-06-18
- Published: 2020-12-24
- Earliest claimed: 2019-06-19
- Assignees: Neusoft Medical Systems Co., Ltd.
- Applicants: Neusoft Medical Systems Co., Ltd.
- Inventors: Feng HUANG
- Web page: https://patentgrep.com/patent/US20200402204A1

## CPC

- G06T 3/4053: based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
- G06T 7/11: Region-based segmentation
- G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining
- G06T 3/4046: using neural networks
- G06T 7/0012: {Biomedical image inspection}
- G06T 7/10: Segmentation; Edge detection (motion-based segmentation G06T7/215)
- G06T 12/00: Tomographic reconstruction from projections
- G06T 12/30: Image post-processing, e.g. metal artefact correction
- G06T 2207/10072: Tomographic images
- G06T 2207/10088: Magnetic resonance imaging [MRI]
- G06T 2207/20056: Discrete and fast Fourier transform, [DFT, FFT]
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30016: Brain
- G06T 2207/30092: Stomach; Gastric
- G06T 2207/30168: Image quality inspection
- G06T 2210/41: Medical
- G06T 2211/424: Iterative
- G06T 2211/441: AI-based methods, deep learning or artificial neural networks
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Methods, devices, systems and apparatus for medical imaging, e.g., Magnetic Resonance (MR) imaging or Computed Tomography (CT) imaging, using neural networks are provided. In one aspect, an imaging method includes: determining a first neural network and a second neural network corresponding to a target imaging task, the first neural network including a first neural network parameter and a first neural network model, the second neural network including a second neural network parameter and a second neural network model, obtaining a reconstructed image by performing reconstruction for down-sampling data of a tissue under test using the first neural network, the target imaging task corresponding to the tissue under test, and obtaining an image output by a second neural network as a target image of the tissue under test by performing an image processing operation corresponding to the target imaging task for the reconstructed image using the second neural network.

## Claims

### Claim 1

An imaging method comprising:
determining a first neural network and a second neural network corresponding to a target imaging task of a plurality of target imaging tasks, wherein the first neural network comprises a first neural network parameter and a first neural network model, and the second neural network comprises a second neural network parameter and a second neural network model, and wherein, for each of the plurality of target imaging tasks, the first neural network comprises a respective first neural network parameter and the second neural network comprises a respective second neural network parameter;
obtaining a reconstructed image by performing reconstruction for down-sampling data of a tissue under test using the first neural network, the target imaging task corresponding to the tissue under test; and
obtaining an image output of the second neural network as a target image of the tissue under test by performing an image processing operation corresponding to the target imaging task for the reconstructed image using the second neural network.

### Claim 2 (depends on 1)

The imaging method of claim 1, wherein obtaining the reconstructed image by performing reconstruction for the down-sampling data of the tissue under test using the first neural network comprises:
obtaining an input image by performing image reconstruction for the down-sampling data of the tissue under test; and
obtaining the reconstructed image by performing reconstruction for the input image using the first neural network.

### Claim 3 (depends on 2)

The imaging method of claim 2, further comprising:
obtaining down-sampling data and full-sampling data of a sample tissue of each sample in a plurality of samples, wherein the sample tissue and the tissue under test are same in type; and
obtaining the first neural network parameter and the second neural network parameter by training the first neural network model and second neural network model according to the down-sampling data and the full-sampling data of the sample tissues of the plurality of the samples and the target image task.

### Claim 4 (depends on 3)

The imaging method of claim 3, wherein obtaining the first neural network parameter and the second neural network parameter by training the first neural network model and the second neural network model comprises:
obtaining an input set, a first annotation set and a second annotation set corresponding to the target imaging task, wherein the input set, the first annotation set and the second annotation set are determined according to the down-sampling data and the full-sampling data of the sample tissues of the plurality of the samples;
obtaining a first output result of the first neural network model by inputting the input set into the first neural network model;
obtaining a second output result of the second neural network model by inputting the first output result into the second neural network model;
obtaining a loss value of a first loss function of the first neural network model based on the first output result and the first annotation set, wherein each of the plurality of target imaging tasks corresponds to a respective first loss function of the first neural network model;
obtaining a loss value of a second loss function of the second neural network model based on the second output result and the second annotation set, wherein each of the plurality of target imaging tasks corresponds to a respective second loss function of the second neural network model; and
stopping training of the first neural network and the second neural network and determining the first neural network parameter and the second neural network parameter in response to at least one of
determining that the first loss function and the second loss function are both converged or
determining that an epoch number of iterative training reaches a preset number, the epoch number being increased by one each time when the plurality of samples iteratively are utilized in the training of the first neural network and the second neural network.

### Claim 5 (depends on 4)

The imaging method of claim 4, wherein obtaining the input set, the first annotation set and the second annotation set corresponding to the target imaging task comprises:
for each sample in the plurality of the samples, obtaining a first reconstructed image by performing image reconstruction for down-sampling data of the sample, the input set corresponding to the target imaging task comprising the plurality of the first reconstructed images;
for each sample in the plurality of the samples, obtaining a second reconstructed image by performing image reconstruction for full-sampling data of the sample, the first annotation set corresponding to the target imaging task comprising the plurality of the second reconstructed images; and
for each second reconstructed image in the plurality of the second reconstructed images, obtaining an image by performing the image processing operation corresponding to the target imaging task for the second reconstructed image, the second annotation set corresponding to the target imaging task comprising the plurality of the images.

### Claim 6 (depends on 2)

The imaging method of claim 2, wherein the down-sampling data of the tissue under test is obtained by performing magnetic resonance (MR) imaging for the tissue under test, and
wherein obtaining the input image by performing image reconstruction for the down-sampling data of the tissue under test comprises:
obtaining the input image by performing reconstruction for down-sampling k-space data of the tissue under test based on inverse Fourier transform or sensitivity encoding.

### Claim 7 (depends on 6)

The imaging method of claim 6, wherein the tissue under test comprises a brain tissue, and the target imaging task comprises grey matter and white matter segmentation.

### Claim 8 (depends on 6)

The imaging method of claim 6, wherein the tissue under test comprises an abdominal tissue, and the target imaging task comprises water and fat separation.

### Claim 9 (depends on 2)

The imaging method of claim 2, wherein the down-sampling data of the tissue under test is obtained by performing Computed Tomography (CT) imaging for the tissue under test, and
wherein obtaining the input image by performing image reconstruction for the down-sampling data of the tissue under test comprises:
obtaining the input image by performing filter back-projection reconstruction for a down-sampling chordal graph of the tissue under test.

### Claim 10 (depends on 9)

The imaging method of claim 9, wherein the tissue under test comprises a lung tissue, and the target imaging task comprises lung tissue segmentation.

### Claim 11

An imaging apparatus for a medical imaging system, the imaging apparatus comprising:
at least one processor; and
at least one non-transitory machine readable storage medium coupled to the at least one processor having machine-executable instructions stored thereon that, when executed by the at least one processor, cause the at least one processor to perform operations comprising:
determining a first neural network and a second neural network corresponding to a target imaging task of a plurality of target imaging tasks, wherein the first neural network comprises a first neural network parameter and a first neural network model, and the second neural network comprises a second neural network parameter and a second neural network model, and wherein, for each of the plurality of target imaging tasks, the first neural network comprises a respective first neural network parameter and the second neural network comprises a respective second neural network parameter;
obtaining a reconstructed image by performing reconstruction for down-sampling data of a tissue under test using the first neural network, the target imaging task corresponding to the tissue under test; and
obtaining an image output of the second neural network as a target image of the tissue under test by performing an image processing operation corresponding to the target imaging task for the reconstructed image using the second neural network.

### Claim 12 (depends on 11)

The imaging apparatus of claim 11, wherein obtaining the reconstructed image by performing reconstruction for the down-sampling data of the tissue under test using the first neural network comprises:
obtaining an input image by performing image reconstruction for the down-sampling data of the tissue under test; and
obtaining the reconstructed image by performing reconstruction for the input image using the first neural network.

### Claim 13 (depends on 12)

The imaging apparatus of claim 12, wherein the operations further comprise:
obtaining down-sampling data and full-sampling data of a sample tissue of each sample in a plurality of samples, wherein the sample tissue and the tissue under test are same in type; and
obtaining the first neural network parameter and the second neural network parameter by training the first neural network model and second neural network model according to the down-sampling data and full-sampling data of the sample tissues of the plurality of the samples and the target imaging task.

### Claim 14 (depends on 13)

The imaging apparatus of claim 13, wherein obtaining the first neural network parameter and the second neural network parameter by training the first neural network model and the second neural network model comprises:
obtaining an input set, a first annotation set and a second annotation set corresponding to the target imaging task, wherein the input set, the first annotation set and the second annotation set are determined by the down-sampling data and full-sampling data of the sample tissues of the plurality of the samples;
obtaining a first output result of the first neural network model by inputting the input set into the first neural network model;
obtaining a second output result of the second neural network model by inputting the first output result into the second neural network model;
obtaining a loss value of a first loss function of the first neural network model based on the first output result and the first annotation set, wherein each of the plurality of target imaging tasks corresponds to a respective first loss function of the first neural network model;
obtaining a loss value of a second loss function of the second neural network model based on the second output result and the second annotation set, wherein each of the plurality of target imaging tasks corresponds to a respective second loss function of the second neural network model; and
stopping training of the first neural network and the second neural network and determining the first neural network parameter and the second neural network parameter in response to at least one of
determining that the first loss function and the second loss function are both converged or
determining that an epoch number of iterative training reaches a preset number, the epoch number being increased by one each time when the plurality of samples iteratively are utilized in the training of the first neural network and the second neural network.

### Claim 15 (depends on 14)

The imaging apparatus of claim 14, wherein obtaining the input set, the first annotation set and the second annotation set corresponding to the target imaging task comprises:
for each sample in the plurality of the samples, obtaining a first reconstructed image by performing image reconstruction for down-sampling data of the sample, the input set corresponding to the target imaging task comprising the plurality of the first reconstructed images;
for each sample in the plurality of the samples, obtaining a second reconstructed image by performing image reconstruction for full-sampling data of the sample, the first annotation set corresponding to the target imaging task comprising the plurality of the second reconstructed images; and
for each second reconstructed image in the plurality of the second reconstructed images, obtaining an image by performing the image processing operation corresponding to the target imaging task for the second reconstructed image, the second annotation set corresponding to the target imaging task comprising the plurality of the images.

### Claim 16 (depends on 12)

The imaging apparatus of claim 12, wherein the medical imaging system comprises a magnetic resonance imaging system, and
wherein obtaining the input image by performing image reconstruction for the down-sampling data of the tissue under test comprises:
obtaining the input image by performing reconstruction for down-sampling k-space data of the tissue under test based on inverse Fourier transform or sensitivity encoding.

### Claim 17 (depends on 16)

The imaging apparatus of claim 16, wherein the target imaging task comprises one of:
grey matter and white matter segmentation for the tissue under test being a brain tissue, and
water and fat separation for the tissue under test being an abdominal tissue.

### Claim 18 (depends on 12)

The imaging apparatus of claim 12, wherein the medical imaging system comprises a CT imaging system, and
wherein obtaining the input image by performing image reconstruction for the down-sampling data of the tissue under test comprises:
obtaining the input image by performing filter back-projection reconstruction for a down-sampling chordal graph of the tissue under test.

### Claim 19 (depends on 18)

The imaging apparatus of claim 18, wherein the tissue under test comprises a lung tissue, and the target imaging task comprises a lung tissue segmentation.

### Claim 20

A medical imaging system comprising:
a medical imaging scanner configured to perform medical scans on tissues with scanning sequences; and
an imaging apparatus coupled to the medical imaging scanner, the imaging apparatus comprising:
at least one processor; and
at least one non-transitory machine readable storage medium coupled to the at least one processor having machine-executable instructions stored thereon that, when executed by the at least one processor, cause the at least one processor to perform operations comprising:
determining a first neural network and a second neural network corresponding to a target imaging task of a plurality of target imaging tasks, wherein the first neural network comprises a first neural network parameter and a first neural network model, and the second neural network comprises a second neural network parameter and a second neural network model, and wherein, for each of the plurality of target imaging tasks, the first neural network comprises a respective first neural network parameter and the second neural network comprises a respective second neural network parameter;
obtaining a reconstructed image by performing reconstruction for down-sampling data of a tissue under test using the first neural network, the target imaging task corresponding to the tissue under test, the down-sampling data of the tissue under test being obtained by performing a medical scan for the tissue under test by the medical imaging scanner; and
obtaining an image output of the second neural network as a target image of the tissue under test by performing an image processing operation corresponding to the target imaging task for the reconstructed image using the second neural network.
