# US 12,008,695 B2 · Methods and systems for translating magnetic resonance images to pseudo computed tomography images

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,008,695 B2
- Document ID: US12008695B2
- Status: Granted
- Kind: B2
- Application number: 17/033,183
- Filed: 2020-09-25
- Published: 2022-03-31
- Granted: 2024-06-11
- Earliest claimed: 2020-09-25
- Assignees: GE Precision Healthcare LLC
- Applicants: GE Precision Healthcare LLC
- Inventors: Sandeep Kaushik; Dattesh Shanbhag; Cristina Cozzini; Florian Wiesinger
- Web page: https://patentgrep.com/patent/US12008695B2

## CPC

- G01R 33/5608: {Data processing and visualization specially adapted for MR, e.g. for feature analysis and pattern recognition on the basis of measured MR data, segmentation of measured MR data, edge contour detection on the basis of measured MR data, for enhancing measured MR data in terms of signal-to-noise ratio by means of noise filtering or apodization, for enhancing measured MR data in terms of resolution by means for deblurring, windowing, zero filling, or generation of gray-scaled images, colour-coded images or images displaying vectors instead of pixels (image data processing or generation, in general G06T)}
- A61B 5/0033: {Features or image-related aspects of imaging apparatus, e.g. for MRI, optical tomography or impedance tomography apparatus; Arrangements of imaging apparatus in a room}
- A61B 5/055: involving electronic [EMR] or nuclear [NMR] magnetic resonance, e.g. magnetic resonance imaging
- A61B 5/7267: {involving training the classification device}
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06T 11/60: Creating or editing images; Combining images with text
- G06T 12/20: Inverse problem, i.e. transformations from projection space into object space
- G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI]
- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 10/82: using neural networks
- A61B 5/0035: {adapted for acquisition of images from more than one imaging mode, e.g. combining MRI and optical tomography}
- A61B 5/7425: {Displaying combinations of multiple images regardless of image source, e.g. displaying a reference anatomical image with a live image}
- A61B 5/743: {Displaying an image simultaneously with additional graphical information, e.g. symbols, charts, function plots}
- G06T 2210/41: Medical
- G06T 2211/441: AI-based methods, deep learning or artificial neural networks
- G06V 2201/033: of skeletal patterns
- G01R: MEASURING ELECTRIC VARIABLES; MEASURING MAGNETIC VARIABLES (indicating correct tuning of resonant circuits H03J3/12)
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Various methods and systems are provided for translating magnetic resonance (MR) images to pseudo computed tomography (CT) images. In one embodiment, a method comprises acquiring an MR image, generating, with a multi-task neural network, a pseudo CT image corresponding to the MR image, and outputting the MR image and the pseudo CT image. In this way, the benefits of CT imaging with respect to accurate density information, especially in sparse regions of bone which exhibit with high dynamic range, may be obtained in an MR-only workflow, thereby achieving the benefits of enhanced soft-tissue contrast in MR images while eliminating CT dose exposure for a patient.

## Claims

### Claim 1

A method, comprising:
acquiring a magnetic resonance (MR) image;
generating, with a multi-task neural network, a pseudo CT image, a bone mask, and a bone Hounsfield unit (HU) image corresponding to the MR image, wherein the pseudo CT image comprises a set of three density classes, the density classes comprising air, tissue, and bone, and wherein the bone HU image includes image values within a bone region of interest in terms of HU; and
outputting the MR image and the pseudo CT image,
wherein the multi-task neural network is trained with a whole image regression loss for the pseudo CT image, a segmentation loss for the bone mask, and a regression loss focused on bone segments for the bone HU image.

### Claim 2 (depends on 1)

The method of claim 1, wherein the multi-task neural network is trained with a focused loss for a region of interest including bone in the MR image.

### Claim 3 (depends on 1)

The method of claim 1, further comprising training the multi-task neural network with a composite loss comprising the whole image regression loss, the segmentation loss, and the regression loss focused on the bone segments, wherein each loss is weighted in the composite loss.

### Claim 4 (depends on 1)

The method of claim 1, further comprising updating the pseudo CT image with the bone HU image, and outputting the updated pseudo CT image with the MR image.

### Claim 5 (depends on 1)

The method of claim 1, wherein the multi-task neural network comprises a U-Net convolutional neural network configured with multiple output layers, and wherein one output layer of the multiple output layers outputs the pseudo CT image.

### Claim 6

A magnetic resonance imaging (MRI) system, comprising:
an MRI scanner;
a display device;
a controller unit communicatively coupled to the MRI scanner and the display device; and
a memory storing executable instructions that when executed cause the controller unit to:
acquire, via the MRI scanner, a magnetic resonance (MR) image;
generate, with a multi-task neural network, a pseudo CT image, a bone mask, and a bone Hounsfield unit (HU) image corresponding to the MR image, wherein the pseudo CT image comprises a set of three density classes, the density classes comprising air, tissue, and bone, and wherein the bone HU image includes image values within a bone region of interest in terms of HU; and
output, to the display device, the MR image and the pseudo CT image,
wherein the multi-task neural network is trained with a whole image regression loss for the pseudo CT image, a segmentation loss for the bone mask, and a regression loss focused on bone segments for the bone HU image.

### Claim 7 (depends on 6)

The MRI system of claim 6, wherein the multi-task neural network is trained with a focused loss for a region of interest including bone in the MR image.

### Claim 8 (depends on 6)

The MRI system of claim 6, the memory further storing executable instructions that when executed cause the controller unit to train the multi-task neural network with a composite loss comprising the whole image regression loss, the segmentation loss, and the regression loss focused on the bone segments, wherein each loss is weighted in the composite loss.

### Claim 9 (depends on 6)

The MRI system of claim 6, the memory further storing executable instructions that when executed cause the controller unit to update the pseudo CT image with the bone HU image, and output, to the display device, the updated pseudo CT image with the MR image.

### Claim 10

A non-transitory computer-readable medium comprising instructions that when executed cause a processor to:
acquire a magnetic resonance (MR) image;
generate, with a multi-task neural network, a pseudo CT image, a bone mask, and a bone Hounsfield unit (HU) image corresponding to the MR image, wherein the pseudo CT image comprises a set of three density classes, the density classes comprising air, tissue, and bone, and wherein the bone HU image includes image values within a bone region of interest in terms of HU; and
output, to a display device, the MR image and the pseudo CT image,
wherein the multi-task neural network is trained with a whole image regression loss for the pseudo CT image, a segmentation loss for the bone mask, and a regression loss focused on bone segments for the bone HU image.

### Claim 11 (depends on 10)

The non-transitory computer-readable medium of claim 10, wherein the multi-task neural network is trained with a focused loss for a region of interest including bone in the MR image.

### Claim 12 (depends on 10)

The non-transitory computer-readable medium of claim 10, wherein the instructions when executed further cause the processor to train the multi-task neural network with a composite loss comprising the whole image regression loss, the segmentation loss, and the regression loss focused on the bone segments, wherein each loss is weighted in the composite loss.

### Claim 13 (depends on 10)

The non-transitory computer-readable medium of claim 10, wherein the instructions when executed further cause the processor to update the pseudo CT image with the bone HU image, and output, to the display device, the updated pseudo CT image with the MR image.

### Claim 14 (depends on 10)

The non-transitory computer-readable medium of claim 10, wherein the multi-task neural network comprises a U-Net convolutional neural network configured with multiple output layers, and wherein one output layer of the multiple output layers outputs the pseudo CT image.

### Claim 15

A method, comprising:
acquiring a magnetic resonance (MR) image;
generating, with a multi-task neural network, a pseudo CT image, a bone mask, and a bone image corresponding to the MR image, wherein the pseudo CT image comprises a set of three density classes, the density classes comprising air, tissue, and bone, wherein the bone mask is obtained via a convolution-sigmoid operation, and the bone image is obtained via a convolution-rectified linear unit (ReLU) operation; and
outputting the MR image and the pseudo CT image.