# US 12,008,690 B2 · Iterative hierarchal network for regulating medical image reconstruction

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,008,690 B2
- Document ID: US12008690B2
- Status: Granted
- Kind: B2
- Application number: 17/155,630
- Filed: 2021-01-22
- Published: 2022-05-26
- Granted: 2024-06-11
- Earliest claimed: 2020-11-25
- Assignees: Siemens Healthineers AG
- Applicants: Siemens Healthcare GmbH
- Inventors: Mahmoud Mostapha; Boris Mailhe; Mariappan S. Nadar; Simon Arberet; Marcel Dominik Nickel
- Web page: https://patentgrep.com/patent/US12008690B2

## CPC

- G06T 12/30: Image post-processing, e.g. metal artefact correction
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/045: Combinations of networks
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/088: Non-supervised learning, e.g. competitive learning
- G06N 3/0895: Weakly supervised learning, e.g. semi-supervised or self-supervised learning
- G06N 3/09: Supervised learning
- G06T 12/20: Inverse problem, i.e. transformations from projection space into object space
- G16H 30/20: for handling medical images, e.g. DICOM, HL7 or PACS
- G16H 30/40: for processing medical images, e.g. editing
- G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems
- G16H 50/70: for mining of medical data, e.g. analysing previous cases of other patients
- G06T 2210/41: Medical
- G06T 2211/424: Iterative
- G06T 2211/441: AI-based methods, deep learning or artificial neural networks
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

For reconstruction in medical imaging, such as reconstruction in MR imaging, an iterative, hierarchal network for regularization may decrease computational complexity. The machine-learned network of the regularizer is unrolled or made iterative. For each iteration, nested U-blocks form a hierarchy so that some of the down-sampling and up-sampling of some U-blocks begin and end with lower resolution data or features, reducing computational complexity.

## Claims

### Claim 1

A method for reconstruction of a medical image in a medical imaging system, the method comprising:
scanning, by the medical imaging system, a patient, the scanning resulting in measurements;
reconstructing, by an image processor, the medical image from the measurements, the reconstructing including a regularizer implemented with a machine-learned network comprising iterative hierarchal convolutional networks, each of the iterative hierarchal convolutional networks including both down-sampling and up-sampling nested within other down-sampling and up-sampling; and
displaying the medical image.

### Claim 2 (depends on 1)

The method of claim 1 wherein scanning comprises scanning with the medical imaging system being a magnetic resonance (MR) scanner and the measurements being k-space measurements.

### Claim 3 (depends on 1)

The method of claim 1 wherein reconstructing comprises reconstructing as an unrolled iterative reconstruction where each of multiple reconstructions in the unrolled iterative reconstruction includes regularization, the regularizer providing the regularization for at least one of the multiple reconstructions.

### Claim 4 (depends on 1)

The method of claim 1 wherein reconstructing comprises reconstructing with the machine-learned network including a feature extraction block comprising one or more convolution layers, the feature extraction block being prior to the iterative hierarchal convolutional networks.

### Claim 5 (depends on 1)

The method of claim 1 wherein reconstructing comprises reconstructing with the machine-learned network wherein the iterative hierarchal convolutional networks have different weights.

### Claim 6 (depends on 1)

The method of claim 1 wherein reconstructing comprises reconstructing with the machine-learned network further comprising a memory network with convolution layers, the memory network applied separately to the outputs of the iterative hierarchal convolutional networks.

### Claim 7 (depends on 6)

The method of claim 6 wherein reconstructing comprises reconstructing with the machine-learned network further comprising an enhancement block of one or more convolutional layers, the enhancement block receiving a concatenation of outputs of the memory network and outputting the medical image as regularized.

### Claim 8 (depends on 1)

The method of claim 1 wherein reconstructing comprises receiving a complex image output by a gradient update, the machine-learned network having two channel input for the complex image, and outputting the medical image with complex values.

### Claim 9 (depends on 1)

The method of claim 1 wherein reconstructing comprises reconstructing with the iterative hierarchal convolutional networks each comprising U-blocks at different levels of the down-sampling and the up-sampling and comprising at least one concatenation connection in parallel with a bottleneck.

### Claim 10 (depends on 9)

The method of claim 9 wherein the U-blocks comprise down-sampling and up-sampling layers and comprise local and global connections.

### Claim 11 (depends on 1)

The method of claim 1 wherein scanning comprise scanning for patient diagnosis as one of different imaging applications for different anatomy and/or disease, and wherein reconstructing comprises application of the machine-learned network independent of the different imaging applications, the machine-learned network having been trained on reconstructions for the different imaging applications.

### Claim 12

A method of machine training for reconstruction in medical imaging, the method comprising:
machine training a neural network with an unrolled arrangement of U-blocks in a sequence as a regularizer for the reconstruction in the medical imaging, the neural network comprising some of the U-blocks nested within other of the U-blocks; and
storing the neural network as machine trained.

### Claim 13 (depends on 12)

The method of claim 12 wherein machine training comprises machine training the neural network with the U-blocks comprising convolutional neural networks with down-sampling and up-sampling.

### Claim 14 (depends on 12)

The method of claim 12 wherein machine training comprises machine training with each of the U-blocks in the sequence having a hierarchy of U-networks.

### Claim 15 (depends on 14)

The method of claim 14 wherein machine training comprises machine training with the hierarchy of U-networks, each of the U-networks in the hierarchy having local and global residual connections.

### Claim 16 (depends on 12)

The method of claim 12 wherein machine training comprises machine training the neural network with the neural network further comprising a convolution network configured to receive outputs from the U-blocks of the sequence and separately process the outputs with shared weights.

### Claim 17 (depends on 12)

The method of claim 12 wherein machine training comprises machine training with ground truth imaging from different medical applications and/or types of imaging.

### Claim 18

A system for reconstruction in medical imaging, the system comprising:
a medical scanner configured to scan a region of a patient, the scan providing scan data;
an image processor configured to reconstruct a representation of the region from the scan data, the image processor configured to reconstruct by application of a machine-learned model in a regularization stage, the machine-learned model comprising a down-sampling and up-sampling first block having multiple down-sampling and up-sampling second blocks; and
a display configured to display an image of the region from the reconstructed representation.

### Claim 19 (depends on 18)

The system of claim 18 wherein the down-sampling and up-sampling first block is one of a sequence of down-sampling and up-sampling first blocks in the regularization stage.

### Claim 20 (depends on 18)

The system of claim 18 wherein the down-sampling and up-sampling first block comprises a first convolutional neural network, and wherein the down-sampling and up-sampling second blocks comprise second convolutional neural networks.