# US 12,182,970 B2 · X-ray imaging restoration using deep learning algorithms

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,182,970 B2
- Document ID: US12182970B2
- Status: Granted
- Kind: B2
- Application number: 17/356,612
- Filed: 2021-06-24
- Published: 2022-12-29
- Granted: 2024-12-31
- Earliest claimed: 2021-06-24
- Assignees: Canon Medical Systems Corporation
- Applicants: Canon Medical Systems Corporation
- Inventors: Yi Hu; Rui Hua; Joseph Manak; John Baumgart; Yu-Bing Chang
- Web page: https://patentgrep.com/patent/US12182970B2

## CPC

- G06T 5/70: Denoising; Smoothing
- A61B 6/42: Arrangements for detecting radiation specially adapted for radiation diagnosis
- A61B 6/5205: {involving processing of raw data to produce diagnostic data}
- A61B 6/585: {Calibration of detector units}
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- G06T 5/60: using machine learning, e.g. neural networks
- G06T 5/73: Deblurring; Sharpening
- G16H 30/20: for handling medical images, e.g. DICOM, HL7 or PACS
- G16H 30/40: for processing medical images, e.g. editing
- A61B 6/5258: {involving detection or reduction of artifacts or noise}
- A61B 6/586: {Detection of faults or malfunction of the device}
- G06T 2207/10072: Tomographic images
- G06T 2207/10116: X-ray image
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30004: Biomedical image processing
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

A general workflow for deep learning based image restoration in X-ray and fluoroscopy/fluorography is disclosed. Higher quality images and lower quality images are generated as training data. This training data can further be categorized by anatomical structure. This training data can be used to train a learned model, such as a neural network or deep-learning neural network. Once trained, the learned model can be used for real-time inferencing. The inferencing can be more further improved by employing a variety of techniques, including pruning the learned model, reducing the precision of the learned mode, utilizing multiple image restoration processors, or dividing a full size image into snippets.

## Claims

### Claim 1

An X-ray diagnosis apparatus, comprising:
an X-ray tube configured to irradiate an imaging region of a subject with X-rays;
an X-ray detector configured to detect the X-rays; and
processing circuitry configured to
acquire, in a first process, a plurality of first X-ray images along a time sequence by detecting X-rays irradiated from the X-ray tube and passed through the subject;
input one of the plurality of first X-ray images to a learned model each time when the one of the plurality of first X-ray images is acquired; and
display, in a second process, an X-ray image, output from the learned model, with less noise and blur each time when the X-ray image is output, wherein the learned model is trained by using a second X-ray image as input learning data and a third X-ray image with less noise and blur than the second X-ray image as output learning data,
wherein the second process of displaying the output X-ray image is performed in parallel with the first process of acquiring the first X-ray images.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein the third X-ray image is obtained by deblurring and denoising the second X-ray image.

### Claim 3 (depends on 1)

The apparatus of claim 1, wherein the second X-ray image is obtained by adding blur and noise to the third X-ray image.

### Claim 4 (depends on 1)

The apparatus of claim 1, wherein the processing circuitry is further configured to prune the learned model after the learned model has been trained.

### Claim 5 (depends on 1)

The apparatus of claim 1, wherein the processing circuitry is further configured to reduce precision of the learned model after the learned model has been trained.

### Claim 6 (depends on 1)

The apparatus of claim 1, wherein the learned model is a deep-learning neural network based on U-net.

### Claim 7 (depends on 1)

The apparatus of claim 1, wherein the first X-ray image, the second X-ray image, and the third X-ray image are of a same anatomic structure.

### Claim 8 (depends on 1)

The apparatus of claim 1, wherein the first X-ray image is a snippet of a full size image.

### Claim 9 (depends on 1)

The apparatus of claim 1, wherein the third X-ray image is a manipulated non-medical dataset, and the second X-ray image is the manipulated non-medical dataset with corruption.

### Claim 10 (depends on 1)

The apparatus of claim 1, wherein a second learned model is trained to take a fourth X-ray image as an input and output the third X-ray image, the second learned model trained using a manipulated non-medical dataset as output learning data, and the manipulated non-medical dataset with corruption as input learning data.

### Claim 11

A method for image restoration, comprising:
acquiring, in a first process, a plurality of first X-ray images along a time sequence by detecting X-rays irradiated from an X-ray tube and passed through a subject using an X-ray detector;
inputting one of the plurality of first X-ray images to a learned model each time when the one of the plurality of first X-ray images is acquired; and
displaying, in a second process, an X-ray image, output from the learned model, with less noise and blur each time when the X-ray image is output, wherein the learned model is trained by using a second X-ray image as input learning data and a third X-ray image with less noise and blur than the second X-ray image as output learning data,
wherein the second process of displaying the output X-ray image is performed in parallel with the first process of acquiring the first X-ray images.

### Claim 12 (depends on 11)

The method of claim 11, wherein the second X-ray image is obtained by deblurring and denoising the first X-ray image.

### Claim 13 (depends on 11)

The method of claim 11, wherein the first X-ray image is obtained by adding blur and noise to the second X-ray image.

### Claim 14 (depends on 11)

The method of claim 11, further comprising pruning the learned model after the learned model has been trained.

### Claim 15 (depends on 11)

The method of claim 11, further comprising reducing precision of the learned model after the learned model has been trained.

### Claim 16 (depends on 11)

The method of claim 11, wherein the third X-ray image is a manipulated non-medical dataset, and the second X-ray image is the manipulated non-medical dataset with corruption.

### Claim 17 (depends on 11)

The method of claim 11, wherein a second learned model is trained to take a fourth X-ray image as an input and output the third X-ray image, the second learned model trained using a manipulated non-medical dataset as output learning data, and the manipulated non-medical dataset with corruption as input learning data.

### Claim 18 (depends on 11)

The method of claim 11, wherein the first X-ray image, the second X-ray image, and the third X-ray image are of a same anatomic structure.

### Claim 19

A non-transitory computer-readable medium including computer-readable instructions that, when executed by a computing system, cause the computing system to sort data by performing a method comprising:
acquiring, in a first process, a plurality of first X-ray images along a time sequence by detecting X-rays irradiated from an X-ray tube and passed through a subject using an X-ray detector;
inputting one of the plurality of first X-ray images to a learned model each time when the one of the plurality of first X-ray images is acquired; and
displaying, in a second process, an X-ray image output from the learned model with less noise and blur each time when the image is output, wherein the learned model is trained by using a second X-ray image as input learning data and a third X-ray image with less noise and blur than the second X-ray image as output learning data,
wherein the second process of displaying the output X-ray image is performed in parallel with the first process of acquiring the first X-ray images.

### Claim 20 (depends on 1)

The apparatus of claim 1, wherein the processing, by the processing circuitry, of inputting the first X-ray image to the learned model and outputting the X-ray image is less computational consuming than a restoring algorithm for generating the third X-ray image.

### Claim 21 (depends on 1)

The apparatus of claim 1, wherein the processing circuitry is further configured to input, to the learned model, the first X-ray image, of a time series of X-ray images, acquired using an X-ray apparatus, wherein a second learned model is trained to take a fourth X-ray image as an input and output the third X-ray image as the output learning data.

### Claim 22 (depends on 1)

The apparatus of claim 1, wherein the processing circuitry is further configured to input, to the learned model, a plurality of a time series of X-ray images including the first X-ray image, and output the X-ray image with less noise and blur.

### Claim 23 (depends on 22)

The apparatus of claim 22, wherein the processing circuitry is further configured to train the learned model using a loss function, which includes a first term being a noise-weighted difference between the time series of X-ray images and a corresponding target image, and a second term being a noise-weighted difference between the first X-ray image and other images in the plurality of the time series of X-ray images.

### Claim 24 (depends on 1)

The apparatus of claim 1, wherein the noise includes at least one of quantum noise and electronic noise, the blur is caused from at least one of light spreading caused by scintillator, a finite focal spot size, and detector binning.