# US 12,182,998 B2 · Self-supervised machine learning for medical image reconstruction

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,182,998 B2
- Document ID: US12182998B2
- Status: Granted
- Kind: B2
- Application number: 17/652,528
- Filed: 2022-02-25
- Published: 2023-08-31
- Granted: 2024-12-31
- Earliest claimed: 2022-02-25
- Assignees: Siemens Healthineers AG
- Applicants: Siemens Healthineers AG
- Inventors: Simon Arberet; Mariappan S. Nadar; Mahmoud Mostapha; Dorin Comaniciu
- Web page: https://patentgrep.com/patent/US12182998B2

## CPC

- G01R 33/5608: {Data processing and visualization specially adapted for MR, e.g. for feature analysis and pattern recognition on the basis of measured MR data, segmentation of measured MR data, edge contour detection on the basis of measured MR data, for enhancing measured MR data in terms of signal-to-noise ratio by means of noise filtering or apodization, for enhancing measured MR data in terms of resolution by means for deblurring, windowing, zero filling, or generation of gray-scaled images, colour-coded images or images displaying vectors instead of pixels (image data processing or generation, in general G06T)}
- G06T 7/0012: {Biomedical image inspection}
- G01R 33/4818: {MR characterised by data acquisition along a specific k-space trajectory or by the temporal order of k-space coverage, e.g. centric or segmented coverage of k-space}
- G16H 30/40: for processing medical images, e.g. editing
- G01R 33/5611: {Parallel magnetic resonance imaging, e.g. sensitivity encoding [SENSE], simultaneous acquisition of spatial harmonics [SMASH], unaliasing by Fourier encoding of the overlaps using the temporal dimension [UNFOLD], k-t-broad-use linear acquisition speed-up technique [k-t-BLAST], k-t-SENSE (structural details of arrays of sub-coils G01R33/3415)}
- G01R 33/56341: {Diffusion imaging}
- G06T 2207/10088: Magnetic resonance imaging [MRI]
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G01R: MEASURING ELECTRIC VARIABLES; MEASURING MAGNETIC VARIABLES (indicating correct tuning of resonant circuits H03J3/12)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

For reconstruction in medical imaging, self-consistency using data augmentation is improved by including data consistency. Artificial intelligence is trained based on self-consistency and data consistency, allowing training without supervision. Fully sampled data and/or ground truth is not needed but may be used. The machine-trained model is less likely to reconstruct images with motion artifacts, and/or the training data may be more easily gathered by not requiring full sampling.

## Claims

### Claim 1

A method of machine training for magnetic resonance (MR) reconstruction in medical imaging, the method comprising:
acquiring MR training data;
generating augmentation data from the MR training data;
machine training a neural network for the MR reconstruction using the MR training data and the augmentation data, wherein the machine training generates an output for each of the pairs of the MR training data and/or augmentation data and uses (1) a self-consistency loss between the respective output pairs and (2) a data-consistency loss and/or data-consistency constraint; and
storing the neural network as machine trained.

### Claim 2 (depends on 1)

The method of claim 1 wherein acquiring comprises acquiring samples of MR k-space data, wherein generating comprises generating samples of MR k-space data, and wherein machine training comprises training the neural network to output an MR image from input of the MR k-space data.

### Claim 3 (depends on 1)

The method of claim 1 wherein generating the augmentation data comprises generating sampling pattern augmentation and/or transposition augmentation.

### Claim 4 (depends on 1)

The method of claim 1 wherein generating the augmentation data comprises separating repetitions for a scan, one of the repetitions being the MR training data and another of the repetitions being the augmentation data.

### Claim 5 (depends on 1)

The method of claim 1 wherein acquiring comprises acquiring the MR training data as datasets without full sampling.

### Claim 6 (depends on 1)

The method of claim 1 wherein machine training comprises calculating the data-consistency loss and/or applying the data-consistency constraint for each of the outputs of the pair.

### Claim 7 (depends on 1)

The method of claim 1 wherein machine training comprises using the data-consistency constraint as integrated into the neural network.

### Claim 8 (depends on 1)

The method of claim 1 wherein machine training comprises using the data-consistency loss.

### Claim 9 (depends on 1)

The method of claim 1 wherein machine training comprises self-supervised machine training.

### Claim 10 (depends on 9)

The method of claim 9 further comprising supervised machine training the neural network after the self-supervised machine training.

### Claim 11 (depends on 1)

The method of claim 1 wherein machine training comprises incorporating the self-consistency loss as a regularization term in supervised machine training.

### Claim 12

A method for reconstruction of a medical image in a medical imaging system, the method comprising:
scanning, by the medical imaging system, a patient, the scanning resulting in measurements;
reconstructing, by an image processor applying a machine-learned model, the medical image from the measurements, the machine-learned model having been trained with both self-consistency and data consistency enforcement using outputs of the machine-learned model when input pairs of sample data, wherein at least one of the sample data is augmented; and
displaying the medical image.

### Claim 13 (depends on 12)

The method of claim 12 wherein scanning comprises magnetic resonance scanning, the measurements comprise k-space data, and the medical image comprises a magnetic resonance image.

### Claim 14 (depends on 12)

The method of claim 12 wherein reconstructing comprises reconstructing by application of the machine-learned model, the machine-learned model having been trained with self-consistency as a self-consistency loss based on data augmentation.

### Claim 15 (depends on 14)

The method of claim 14 wherein reconstructing comprises reconstructing by the application of the machine-learned model, the machine-learned model having been trained with self-supervised training using the self-consistency loss where the data augmentation was sampling pattern augmentation and/or transpose augmentation.

### Claim 16 (depends on 14)

The method of claim 14 wherein reconstructing comprises reconstructing by the application of the machine-learned model, the machine-learned model having been trained where the data augmentation was division of multiple scan repetitions into separate inputs.

### Claim 17 (depends on 12)

The method of claim 12 wherein reconstructing comprises reconstructing by application of the machine-learned model, the machine-learned model having been trained with the data consistency being a data consistency constraint in an architecture of the machine-learned model or a data consistency loss.

### Claim 18 (depends on 12)

The method of claim 12 wherein reconstructing comprises reconstructing by application of the machine-learned model, the machine-learned model having been trained with both self-consistency and data consistency in a first optimization and then supervised learning in a second optimization.

### Claim 19

A system for reconstruction in medical imaging, the system comprising:
a medical scanner configured to scan a patient;
an image processor configured to reconstruct a representation of the patient, the image processor configured to reconstruct by application of a machine-learned model having been trained for the reconstruction based on self-supervised learning with data consistency and self-consistency enforcement using outputs of the machine-learned model when input pairs of sample data, wherein at least one of the sample data is augmented; and
a display configured to display an image of the patient, the image formed from the representation.