# US 2021/0342638 A1 · X-ray image synthesis from ct images for training nodule detection systems

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0342638 A1
- Document ID: US20210342638A1
- Status: Published application
- Kind: A1
- Application number: 15/929,427
- Filed: 2020-05-01
- Published: 2021-11-04
- Earliest claimed: 2020-05-01
- Assignees: Siemens Healthcare GmbH
- Applicants: Siemens Healthcare GmbH
- Inventors: Florin-Cristian Ghesu; Siqi Liu; Arnaud Arindra Adiyoso; Sasa Grbic; Marvin Teichmann
- Web page: https://patentgrep.com/patent/US20210342638A1

## CPC

- G06T 7/0012: {Biomedical image inspection}
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F 18/25: Fusion techniques
- G06N 20/00: Machine learning
- G06T 3/60: Rotation of whole images or parts thereof
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 2207/10072: Tomographic images
- G06T 2207/10081: Computed x-ray tomography [CT]
- G06T 2207/10116: X-ray image
- G06T 2207/20081: Training; Learning
- G06T 2207/30064: Lung nodule
- G06T 2207/30168: Image quality inspection
- G06V 2201/032: of protuberances, polyps nodules, etc.
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Systems and methods for generating synthesized medical images for training a machine learning based network are provided. An input medical image in a first modality is received. The input medical image comprises a nodule region for each of one or more nodules and a remaining region. The input medical image comprises an annotation for each of the one or more nodules. A synthesized medical image in a second modality is generated from the input medical image. The synthesized medical image comprises the annotation for each of the one or more nodules. A synthesized nodule image of each of the nodule regions and synthesized remaining image of the remaining region are generated in the second modality. It is determined whether each particular nodule of the one or more nodules is visible in the synthesized medical image based on at least one of the synthesized nodule image for the particular nodule and the synthesized remaining image. In response to determining that at least one nodule of the one or more nodules is not visible in the synthesized medical image, the annotation for the at least one not visible nodule is removed from the synthesized nodule image.

## Claims

### Claim 1

A computer implemented method comprising:
receiving an input medical image in a first modality, the input medical image comprising a nodule region for each of one or more nodules and a remaining region, the input medical image comprising an annotation for each of the one or more nodules;
generating, from the input medical image, a synthesized medical image in a second modality, the synthesized medical image comprising the annotation for each of the one or more nodules;
generating, in the second modality, 1) a synthesized nodule image of each of the nodule regions and 2) a synthesized remaining image of the remaining region;
determining whether each particular nodule of the one or more nodules is visible in the synthesized medical image based on at least one of the synthesized nodule image for the particular nodule and the synthesized remaining image; and
in response to determining that at least one nodule of the one or more nodules is not visible in the synthesized medical image, removing the annotation for the at least one not visible nodule from the synthesized nodule image.

### Claim 2 (depends on 1)

The method of claim 1, wherein determining whether each particular nodule of the one or more nodules is visible in the synthesized medical image based on at least one of the synthesized nodule image for the particular nodule and the synthesized remaining image comprises:
comparing an intensity of pixels in the synthesized remaining image to a threshold.

### Claim 3 (depends on 1)

The method of claim 1, wherein determining whether each particular nodule of the one or more nodules is visible in the synthesized medical image based on at least one of the synthesized nodule image for the particular nodule and the synthesized remaining image comprises:
determining a change between an intensity at a center of the particular nodule in the synthesized remaining image and an intensity at the center of the particular nodule in the synthesized nodule image; and
comparing the intensity change to a threshold.

### Claim 4 (depends on 1)

The method of claim 1, wherein determining whether each particular nodule of the one or more nodules is visible in the synthesized medical image based on at least one of the synthesized nodule image for the particular nodule and the synthesized remaining image comprises:
determining a change between an average intensity of pixels within half of a radius of the particular nodule in the synthesized remaining image and an average intensity of pixels within half of the radius of the particular nodule in the synthesized nodule image; and
comparing the change to a threshold.

### Claim 5 (depends on 1)

The method of claim 1, wherein generating, from the input medical image, a synthesized medical image in a second modality comprises:
increasing an intensity of pixels in the synthesized nodule images by a highlighting factor.

### Claim 6 (depends on 1)

The method of claim 1, wherein generating, from the input medical image, a synthesized medical image in a second modality comprises:
positioning a previously acquired synthesized nodule image in the synthesized medical image.

### Claim 7 (depends on 1)

The method of claim 1, wherein generating, from the input medical image, a synthesized medical image in a second modality comprises:
rotating the input medical image; and
generating the synthesized medical image from the rotated input medical image.

### Claim 8 (depends on 1)

The method of claim 1, further comprising:
training a machine learning based network based on the synthesized medical image.

### Claim 9 (depends on 1)

The method of claim 1, wherein receiving an input medical image in a first modality comprises:
generating the input medical image to include one or more synthesized nodules.

### Claim 10 (depends on 1)

The method of claim 1, wherein the first modality is computed tomography and the second modality is x-ray.

### Claim 11

An apparatus comprising:
means for receiving an input medical image in a first modality, the input medical image comprising a nodule region for each of one or more nodules and a remaining region, the input medical image comprising an annotation for each of the one or more nodules;
means for generating, from the input medical image, a synthesized medical image in a second modality, the synthesized medical image comprising the annotation for each of the one or more nodules;
means for generating, in the second modality, 1) a synthesized nodule image of each of the nodule regions and 2) a synthesized remaining image of the remaining region;
means for determining whether each particular nodule of the one or more nodules is visible in the synthesized medical image based on at least one of the synthesized nodule image for the particular nodule and the synthesized remaining image; and
means for in response to determining that at least one nodule of the one or more nodules is not visible in the synthesized medical image, removing the annotation for the at least one not visible nodule from the synthesized nodule image.

### Claim 12 (depends on 11)

The apparatus of claim 11, wherein the means for determining whether each particular nodule of the one or more nodules is visible in the synthesized medical image based on at least one of the synthesized nodule image for the particular nodule and the synthesized remaining image comprises:
means for comparing an intensity of pixels in the synthesized remaining image to a threshold.

### Claim 13 (depends on 11)

The apparatus of claim 11, wherein the means for determining whether each particular nodule of the one or more nodules is visible in the synthesized medical image based on at least one of the synthesized nodule image for the particular nodule and the synthesized remaining image comprises:
means for determining a change between an intensity at a center of the particular nodule in the synthesized remaining image and an intensity at the center of the particular nodule in the synthesized nodule image; and
means for comparing the intensity change to a threshold.

### Claim 14 (depends on 11)

The apparatus of claim 11, wherein the means for determining whether each particular nodule of the one or more nodules is visible in the synthesized medical image based on at least one of the synthesized nodule image for the particular nodule and the synthesized remaining image comprises:
means for determining a change between an average intensity of pixels within half of a radius of the particular nodule in the synthesized remaining image and an average intensity of pixels within half of the radius of the particular nodule in the synthesized nodule image; and
means for comparing the change to a threshold.

### Claim 15 (depends on 11)

The apparatus of claim 11, wherein the means for generating, from the input medical image, a synthesized medical image in a second modality comprises:
means for increasing an intensity of pixels in the synthesized nodule images by a highlighting factor.

### Claim 16

A non-transitory computer readable medium storing computer program instructions, the computer program instructions when executed by a processor cause the processor to perform operations comprising:
receiving an input medical image in a first modality, the input medical image comprising a nodule region for each of one or more nodules and a remaining region, the input medical image comprising an annotation for each of the one or more nodules;
generating, from the input medical image, a synthesized medical image in a second modality, the synthesized medical image comprising the annotation for each of the one or more nodules;
generating, in the second modality, 1) a synthesized nodule image of each of the nodule regions and 2) a synthesized remaining image of the remaining region;
determining whether each particular nodule of the one or more nodules is visible in the synthesized medical image based on at least one of the synthesized nodule image for the particular nodule and the synthesized remaining image; and
in response to determining that at least one nodule of the one or more nodules is not visible in the synthesized medical image, removing the annotation for the at least one not visible nodule from the synthesized nodule image.

### Claim 17 (depends on 16)

The non-transitory computer readable medium of claim 16, wherein generating, from the input medical image, a synthesized medical image in a second modality comprises:
positioning a previously acquired synthesized nodule image in the synthesized medical image.

### Claim 18 (depends on 16)

The non-transitory computer readable medium of claim 16, wherein generating, from the input medical image, a synthesized medical image in a second modality comprises:
rotating the input medical image; and
generating the synthesized medical image from the rotated input medical image.

### Claim 19 (depends on 16)

The non-transitory computer readable medium of claim 16, further comprising:
training a machine learning based network based on the synthesized medical image.

### Claim 20 (depends on 16)

The non-transitory computer readable medium of claim 16, wherein the first modality is computed tomography and the second modality is x-ray.
