# US 11,308,587 B2 · Learning method of generative adversarial network with multiple generators for image denoising

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,308,587 B2
- Document ID: US11308587B2
- Status: Granted
- Kind: B2
- Application number: 17/288,933
- Filed: 2020-09-29
- Published: 2022-03-24
- Granted: 2022-04-19
- Earliest claimed: 2019-10-23
- Assignees: University of Shanghai for Science and Technology
- Applicants: University of Shanghai for Science and Technology; Yunwu Netlink (Suzhou) Intelligent Technology Co., Ltd
- Inventors: Songwen Pei; Jing Fan
- Web page: https://patentgrep.com/patent/US11308587B2

## CPC

- G06T 5/70: Denoising; Smoothing
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/045: Combinations of networks
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/047: Probabilistic or stochastic networks
- G06N 3/0475: Generative networks
- G06N 3/088: Non-supervised learning, e.g. competitive learning
- G06N 3/09: Supervised learning
- G06N 3/094: Adversarial learning
- G06T 5/60: using machine learning, e.g. neural networks
- G06T 2207/10081: Computed x-ray tomography [CT]
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30096: Tumor; Lesion
- Y02T 10/40: Engine management systems
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- Y02T: CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION

## Abstract

The present invention relates to a learning method of generative adversarial network (GAN) with multiple generators for image denoising, and provides a generative adversarial network with three generators. Such generators are used for removing Poisson noise, Gaussian blur noise and distortion noise respectively to improve the quality of low-dose CT (LDCT) images; the generators adopt the residual network structure. The mapped short connection used in the residual network can avoid the vanishing gradient problem in a deep neural network and accelerate the network training; the training of GAN is always a difficult problem due to the unreasonable measure between the generative distribution and real distribution. The present invention can stabilize training and enhance the robustness of training models by limiting the spectral norm of a weight matrix.

## Claims

### Claim 1

A learning method of generative adversarial network (GAN) with multiple generators for image denoising, wherein a LDCT image represents a low-dose CT image, and a NDCT image represents a normal-dose CT image; characterized in that it specifically comprises the following steps:
1) Constructing an adversarial network structure: The adversarial network comprises three different generators G1, G2, G3 and a discriminator D, wherein G1, G2 and G3 are generators for removing Gaussian blur noise, Poisson quantum noise and distortion noise respectively, and the discriminator D is used for distinguishing real and generated images; both the generators and the discriminator adopt the neural network structure, and G1, G2 and G3 adopt the residual network structure, with independent parameters;
2) Generators: The LDCT image, serving as a sample, is correspondingly inputted into three generators according to the weights λ1, λ2 and λ3 of the generators, wherein the generated image outputted by the G1 generator enters the discriminator D and preprocessed λ1D23 for calculation of pixel loss, and also adds the generated images of G2 and G3 to generate a new generator image, which, together with the NDCT image, is used as the input of the discriminator D; the same is true for the other two generators;
D23 is a sample of image distribution with Poisson quantum noise and distortion added to the NDCT image where G1 is a generator for Gaussian blur denoising; D13 is a sample of image distribution with Gaussian blur and distortion added to the NDCT image where G2 is a generator for Poisson quantum denoising; D12 is a sample of image distribution with Gaussian blur and Poisson quantum noise added to the NDCT image where G3 is a generator for distortion denoising;
3) Discriminator: To keep the image generated by the generator and its corresponding NDCT image as approximate as possible so that the discriminator is successfully cheated, the adversarial loss function La (consisting of the generator loss and the discriminator loss) is minimized, and the loss function of the whole discriminator is as follows:
Loss
=
L
a
+
L
p
⁢
1
+
L
p
⁢
2
+
L
p
⁢
3
L
a
=
sup

f

L
≤
1
⁢
∑
i
=
1
3
⁢
[
E
s
∼
λ
i
⁢
P
r
⁢
f
⁡
(
s
)
⁢
-
⁢
E
t
∼
P
gi
⁢
f
⁡
(
t
)
]
L
p
⁢
1
=
μ
1
⁢
E
⁡
[

G
1
⁡
(
λ
1
⁢
z
)
-
λ
1
⁢
D
2
⁢
3

2
]
L
p
⁢
2
=
μ
2
⁢
E
⁡
[

G
2
⁡
(
λ
2
⁢
z
)
-
λ
2
⁢
D
1
⁢
3

2
]
L
p
⁢
3
=
μ
3
⁢
E
⁡
[

G
3
⁡
(
λ
3
⁢
z
)
-
λ
3
⁢
D
1
⁢
2

2
]
where La represents the adversarial loss function; λ1+λ2+λ3=1; ƒ(x) is the 1-Lipschitz function; Es˜λiPrƒ(s) represents the mean of ƒ(s) when s is an authentic sample; Et˜Pgi(t) represents the mean of ƒ(t) when t is a sample generated by the generator; ∥ƒ∥L≤1 represents the Lipschitz constant ∥ƒ∥L of ƒ(x) no more than 1; Pr is the distribution of NDCT image, λi Pr refers to selection of λi times the distributions of NDCT image; Pgi represents the distribution of CT image generated by the ith generator; s and t are samples of λi Pr and Pgi respectively; ∥ ∥ represents the Frobenius-2 norm; z represents the distribution of LDCT image; Lp1, Lp2 and Lp3 represent the pixel losses between the image generated by each generator and the corresponding preprocessed noise-added label image, while μ1, μ2 and μ3 are predefined weights for different pixel losses, with the effect of weighing the pixel losses;
4) The established adversarial network structure is trained, and then the actually sampled LDCT image is learned and processed.