# US 12,602,798 B2 · Method and apparatus for generating subject-specific magnetic resonance angiography images from other multi-contrast magnetic resonance images

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,602,798 B2
- Document ID: US12602798B2
- Status: Granted
- Kind: B2
- Application number: 18/242,982
- Filed: 2023-09-06
- Published: 2025-03-06
- Granted: 2026-04-14
- Earliest claimed: 2023-09-06
- Assignees: University of Leeds
- Applicants: University of Leeds
- Inventors: Yan Xia; Nishant Ravikumar; Alejandro Federico Frangi
- Web page: https://patentgrep.com/patent/US12602798B2

## CPC

- G06T 7/32: using correlation-based methods
- G06T 3/40: Scaling of whole images or parts thereof, e.g. expanding or contracting
- G16H 30/40: for processing medical images, e.g. editing
- G06T 2207/10088: Magnetic resonance imaging [MRI]
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/20212: Image combination
- G06T 2207/30016: Brain
- G06T 2207/30101: Blood vessel; Artery; Vein; Vascular
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

There is provided a computer-implemented method for synthesising magnetic resonance angiography (MRA) images from other types of inputted magnetic resonance (MR) images, in a subject-specific manner, the method comprising providing a conditional generative adversarial network (cGAN) that learns a combined latent representation of the inputted magnetic resonance images for each subject and learns to transform this combined latent representation to a magnetic resonance angiography image corresponding to that subject, providing a plurality of magnetic resonance (MR) images as input into the cGAN, and outputting a plurality of MRA images from the cGAN based on the plurality of inputted MR images.

## Claims

### Claim 1

A computer-implemented method for synthesising magnetic resonance angiography (MRA) images from other types of inputted magnetic resonance (MR) images, in a subject-specific manner, the method comprising:
providing a conditional generative adversarial network (cGAN) that learns a combined latent representation of the inputted magnetic resonance images for each subject and learns to transform this combined latent representation to a magnetic resonance angiography image corresponding to that subject;
providing a plurality of magnetic resonance (MR) images as input into the cGAN; and
outputting a plurality of MRA images from the cGAN based on the plurality of inputted MR images;
wherein the cGAN comprises:
a Generator network that learns a combined latent representation of the inputted MR images for each subject and learns to transform the latent representation to MRA images for each corresponding subject,
a Discriminator network operating at a single or multiple spatial scales/resolution, to compete with the Generator network and improve the performance of the Generator network at synthesising MRA images using a loss function, and
wherein the loss function comprises a plurality of 2D projection MRA domain-specific perceptual loss terms that act on maximum intensity projection images along three different axes: axial, coronal and sagittal.

### Claim 2 (depends on 1)

The method of claim 1, wherein the Generator network comprises a convolutional encoder-decoder network architecture operable to learn to transform any combination of inputted MR images of a given subject to an MRA image specific to that subject.

### Claim 3 (depends on 1)

The method of claim 1, wherein the Discriminator network is used in combination with any pre-trained convolutional neural network to determine perceptual losses that are used to train the cGAN.

### Claim 4 (depends on 1)

The method of claim 1, wherein data inputted to the Discriminator network to compute feature matching losses, that are used to train the cGAN, are any combination of real and/or synthesised MRA images in their native 3D domain, or their 2D projection domains.

### Claim 5 (depends on 1)

The method of claim 1, wherein the Discriminator network comprises a Multi-scale Discriminator network comprising two sub-networks, wherein a first sub-network operates at a higher scale/resolution and wherein a second sub-network operates at a lower spatial scale/resolution, wherein the two sub-networks are trained using a modified version of the least-squares adversarial loss function.

### Claim 6 (depends on 5)

The method of claim 5, wherein an overall loss function, overall, comprises:
overall=cGAN(G,D)+recon (2):
wherein the Loss function, cGAN(G, Dk), of the Discriminator network comprises:
ℒ
cGAN
(
G
,
D
k
)
=
-
𝔼
I
N
,
y
[
(
D
k
(
I
n
,
y
)
-
1
)
2
]
-
𝔼
I
N
[
D
k
(
I
n
,
G
⁡
(
I
n
)
)
2
]
(
6
)
and
wherein the reconstruction Loss, recon, comprises:
ℒ
recon
=
𝔼
I
N
,
y
[

y
-
G
⁡
(
I
n
)

1
]
⁢
or
⁢
ℒ
recon
=
𝔼
I
N
,
y
[

y
-
G
⁡
(
I
n
)

2
]
.
(
3
)

### Claim 7 (depends on 1)

The method of claim 1, wherein the Discriminator network Dk comprises a multi-scale Discriminator network and wherein the method further comprises use of a feature matching loss FM3D that minimises a difference in feature representations at intermediate layers of the multi-scale Discriminator network DR and is formulated as:
ℒ
FM
⁢
3
⁢
D
(
G
,
D
k
)
=
𝔼
I
N
,
y
⁢
∑
l
=
1
L
⁢
1
N
l
[

D
k
i
(
I
n
,
y
)
-
D
k
i
(
I
n
,
G
⁡
(
I
n
)
)

1
]
.
(
7
)

### Claim 8 (depends on 1)

The method of claim 1, wherein the respective 2D projection perceptual loss terms are given by the equations:
ℒ
PER
⁢
2
⁢
D
-
axial
(
G
,
V
)
=
𝔼
I
N
,
y
⁢
∑
f
=
1
F
⁢
1
M
f
[

V
f
(
p
axial
(
y
)
)
-
V
f
(
p
axial
(
G
⁡
(
I
n
)
)
)

1
]
,
(
9
)
ℒ
PER
⁢
2
⁢
D
-
coronal
(
G
,
V
)
=
𝔼
I
N
,
y
⁢
∑
f
=
1
F
⁢
1
M
f
[

V
f
(
p
coronal
(
y
)
)
-
V
f
(
p
coronal
(
G
⁡
(
I
n
)
)
)

1
]
,
(
10
)
ℒ
PER
⁢
2
⁢
D
-
sagittal
(
G
,
V
)
=
𝔼
I
N
,
y
⁢
∑
f
=
1
F
⁢
1
M
f
[

V
f
(
p
sagittal
(
y
)
)
-
V
f
(
p
sagittal
(
G
⁡
(
I
n
)
)
)

1
]
.
(
11
)

### Claim 9 (depends on 1)

The method of claim 1, wherein the loss function further comprises a 3D volume component, said 3D loss function component comprising:
ℒ
P
⁢
E
⁢
R
⁢
3
⁢
D
(
G
,
V
)
=
𝔼
I
N
,
y
⁢
∑
f
=
1
F
1
M
f
[

V
f
(
y
)
-
V
f
(
G
⁡
(
I
n
)
)

1
]
.
(
8
)

### Claim 10 (depends on 1)

The method of claim 1, wherein the loss function comprises feature matching loss terms and a perceptual loss terms, and wherein these are combined to formulate a combined reconstruction loss recon comprising the equation:
ℒ
recon
=
λ
1
[
∑
k
=
1
2
⁢
ℒ
FM
⁢
3
⁢
D
(
G
,
D
k
)
]
+
λ
2
⁢
ℒ
PER
⁢
3
⁢
D
(
G
,
V
)
+
λ
3
⁢
ℒ
PER
⁢
2
⁢
D
-
axial
(
G
,
V
)
+
λ
4
⁢
ℒ
PER
⁢
2
⁢
D
-
coronal
(
G
,
V
)
+
λ
5
⁢
ℒ
PER
⁢
2
⁢
D
-
sagittal
(
G
,
V
)
(
12
)
where, λ1, λ2, λ3, λ4 and λ5 represent weights that control the influence of each feature matching or perceptual loss term in recon on an overall learning process of the method.

### Claim 11 (depends on 1)

The method of claim 1, wherein a complete loss function used to train the cGAN in an end-to-end fashion is given by the equation:
ℒ
complete
=
min
G
[
[
max
D
⁢
1
,
D
⁢
2
∑
k
=
1
2
⁢
ℒ
cGAN
(
G
,
D
k
)
]
+
ℒ
recon
]
.
(
13
)

### Claim 12 (depends on 1)

A non-transitory computer readable medium comprising instructions, which, when carried out by one or more processors, cause the one or more processors to carry out the method of claim 1.

### Claim 13 (depends on 1)

A method of developing medical devices or medicines comprising:
deriving generated MRA images for testing medical devices or medicines under development from real MRI images using the method of claim 1; and
testing the medical devices or medicines using the generated MRA images.