# US 2024/0005451 A1 · Methods and systems for super-resolution with progressive sub-voxel up-sampling

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0005451 A1
- Document ID: US20240005451A1
- Status: Published application
- Kind: A1
- Application number: 17/810,271
- Filed: 2022-06-30
- Published: 2024-01-04
- Earliest claimed: 2022-06-30
- Assignees: GE Precision Healthcare LLC
- Applicants: GE Precision Healthcare LLC
- Inventors: Rohan Keshav Patil; Sudhanya Chatterjee
- Web page: https://patentgrep.com/patent/US20240005451A1

## CPC

- G06T 3/4053: based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
- G06T 3/4076: using the original low-resolution images to iteratively correct the high-resolution images
- G06T 3/4046: using neural networks
- G16H 30/40: for processing medical images, e.g. editing
- G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

Various methods and systems are provided for generating super-resolution images. In one embodiment, a method comprises: progressively up-sampling an input image to generate a super-resolution output image by: generating N intermediate images based on the input image, where N is equal to at least one, including a first intermediate image by providing the input image to a deep neural network, where a resolution of the first intermediate image is a multiple of a resolution of the input image, higher than the resolution of the input image, and can be any positive real value and not necessarily an integer value; generating the super-resolution output image based on the N intermediate images, the super-resolution output image having a resolution higher than a respective resolution of each intermediate image of the N intermediate images and the resolution of the input image; and displaying the super-resolution output image via a display device.

## Claims

### Claim 1

A method, comprising:
progressively up-sampling an input image to generate a super-resolution output image by:
generating N intermediate images based on the input image, where N is equal to at least one, including a first intermediate image by providing the input image to a deep neural network, where a resolution of the first intermediate image is a multiple of a resolution of the input image, higher than the resolution of the input image, and can be any positive real value and not necessarily an integer value;
generating the super-resolution output image based on the N intermediate images, the super-resolution output image having a resolution higher than a respective resolution of each intermediate image of the N intermediate images and the resolution of the input image;
and
displaying the super-resolution output image via a display device and/or storing the super-resolution output image to a computer memory.

### Claim 2 (depends on 1)

The method of claim 1, wherein generating the super-resolution output image based on the first intermediate image includes providing the first intermediate image to the deep neural network.

### Claim 3 (depends on 1)

The method of claim 1, wherein generating the super-resolution output image based on the N intermediate images includes:
generating a second intermediate image directly from the first intermediate image by providing the first intermediate image to the deep neural network, where a resolution of the second intermediate image is a multiple of the resolution of the input image, higher than the resolution of the input image, and can be any positive real value and not necessarily an integer value; and
generating the super-resolution output image based on the second intermediate image.

### Claim 4 (depends on 3)

The method of claim 3, wherein generating the super-resolution output image based on the second intermediate image includes:
providing the second intermediate image to the deep neural network.

### Claim 5 (depends on 3)

The method of claim 3, wherein generating the super-resolution output image based on the second intermediate image includes:
generating a third intermediate image directly from the second intermediate image by providing the second intermediate image to the deep neural network, where a resolution of the third intermediate image is a multiple of the resolution of the input image, higher than the resolution of the input image, and can be any positive real value and not necessarily an integer value; and
generating the super-resolution output image based on the third intermediate image.

### Claim 6 (depends on 1)

The method of claim 1, wherein the first intermediate image is first in a sequence of the N intermediate images.

### Claim 7 (depends on 6)

The method of claim 6, wherein each intermediate image of the N intermediate images has a respective resolution higher than each preceding intermediate image of the N intermediate images in the sequence of the N intermediate images.

### Claim 8 (depends on 6)

The method of claim 6, wherein a respective resolution of each intermediate image of the N intermediate images is a different multiple of the resolution of the input image relative to the respective resolution of each other intermediate image of the N intermediate images.

### Claim 9 (depends on 8)

The method of claim 8, wherein the N intermediate images includes the first intermediate image, a second intermediate image based on the first intermediate image, and a third intermediate image based on the second intermediate image.

### Claim 10 (depends on 9)

The method of claim 9, wherein the respective resolution of the first intermediate image is equal to the resolution of the input image multiplied by an amount within a first range of positive real values and is not necessarily an integer value, and the respective resolution of the third intermediate image is equal to the resolution of the input image multiplied by an amount within a second range of positive real values and is not necessarily an integer value.

### Claim 11 (depends on 1)

The method of claim 1, further comprising displaying the input image, the N intermediate images, and the super-resolution output image concurrently via the display device.

### Claim 12 (depends on 1)

The method of claim 1, wherein the deep neural network is a convolutional neural network.

### Claim 13

A method, comprising:
acquiring a medical image;
generating N intermediate images based on the medical image, where N is equal to at least one, including a first intermediate image via a feature learner and an up-sampling model trained on images acquired with an imaging modality of the medical image, where a resolution of the first intermediate image is a multiple of a resolution of the medical image, higher than the resolution of the medical image, and can be any positive real value and not necessarily an integer value; and
generating a super-resolution output image based on the N intermediate images via the feature learner and the up-sampling model.

### Claim 14 (depends on 13)

The method of claim 13, wherein the imaging modality is magnetic resonance imaging (MRI).

### Claim 15 (depends on 14)

The method of claim 14, wherein generating the super-resolution output image based on the N intermediate images includes zero-padding a k-space of MR image data of the first intermediate image and providing the first intermediate image to the feature learner and the up-sampling model.

### Claim 16 (depends on 14)

The method of claim 14, wherein generating the super-resolution output image based on the N intermediate images includes generating a second intermediate image from the first intermediate image by providing the first intermediate image to the feature learner and the up-sampling model, where a k-space of image data of the first intermediate image is zero-padded by the up-sampling model, and providing the second intermediate image to the feature learner and the up-sampling model to generate the super-resolution output image.

### Claim 17 (depends on 16)

The method of claim 16, wherein a resolution of the second intermediate image is a multiple of the resolution of the medical image, is greater than the resolution of the first intermediate image, and can be any positive real value and not necessarily an integer value.

### Claim 18

A system, comprising:
a memory storing a deep neural network, and instructions; and
a processor, wherein the processor is communicably coupled to the memory, and when executing the instructions, is configured to:
generate N intermediate images based on an input image, where N is equal to at least one, including a first intermediate image by providing the input image to the deep neural network, where a resolution of the first intermediate image is a multiple of a resolution of the input image, higher than the resolution of the input image, and can be any positive real value and not necessarily an integer value; and
generate a super-resolution output image based on the N intermediate images, the super-resolution output image having a resolution higher than a respective resolution of each intermediate image of the N intermediate images and the resolution of the input image.

### Claim 19 (depends on 18)

The system of claim 18, further comprising instructions stored in the memory that when executed, cause the processor to:
up-sample the first intermediate image via the deep neural network.

### Claim 20 (depends on 18)

The system of claim 18, further comprising instructions stored in the memory that when executed, cause the processor to:
zero-pad a k-space of image data of the first intermediate image prior to generating the super-resolution output image based on the first intermediate image.
