# US 5,974,165 A · Automated method and system for the alignment and correlation of images from two different modalities

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 5,974,165 A
- Document ID: US5974165A
- Status: Granted
- Kind: A
- Application number: 08/523,210
- Filed: 1995-09-05
- Granted: 1999-10-26
- Earliest claimed: 1993-11-30
- Assignees: Arch Development Corporation
- Inventors: Maryellen L. Giger; Chin-Tu Chen; Samuel Armato; Kunio Doi
- Web page: https://patentgrep.com/patent/US5974165A

## CPC

- G06T 7/0012: {Biomedical image inspection}
- G06T 3/147: using affine transformations
- G06T 7/12: Edge-based segmentation
- G06T 7/155: involving morphological operators
- G06T 7/66: of image moments or centre of gravity
- G06T 2207/10116: X-ray image
- G06T 2207/30061: Lung
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A method and system for the computerized registration of radionuclide images with radiographic images, including generating image data from radiographic and radionuclide images of the thorax. Techniques include contouring the lung regions in each type of chest image, scaling and registration of the contours based on location of lung apices, and superimposition after appropriate shifting of the images. Specific applications are given for the automated registration of radionuclide lungs scans with chest radiographs. The method in the example given yields a system that spatially registers and correlates digitized chest radiographs with V/Q scans in order to correlate V/Q functional information with the greater structural detail of chest radiographs. Final output could be the computer-determined contours from each type of image superimposed on any of the original images, or superimposition of the radionuclide image data, which contains high activity, onto the radiographic chest image.

## Claims

### Claim 1

A method of correlating images from two modalities, comprising:
obtaining a first image containing an anatomic feature using a first modality;
obtaining a second image containing said anatomic feature using a second modality;
detecting a first feature contour of said anatomic feature in said first image iteratively using estimated contours derived using a plurality of successive gray-level thresholding operations at a respective plurality of gray-level thresholds, including:
thresholding said first image using one of said plurality of gray-level thresholds,
determining whether to alter pixels in said first image, and
thresholding said first image using a subsequent one of said gray-level thresholds after said determining step;
detecting a second feature contour of said anatomic feature in said second image; and
superposing said first feature contour and said second feature contour.

### Claim 2 (depends on 1)

A method as recited in claim 1, further comprising scaling said first feature contour.

### Claim 3 (depends on 1)

A method as recited in claim 1, wherein performing iterative gray-level thresholding comprises:
obtaining a global gray-level histogram of said first image including said anatomic feature; and
obtaining a plurality of global gray-level threshold values using said global gray-level histogram.

### Claim 4 (depends on 1)

A method as recited in claim 1, wherein performing iterative gray-level thresholding comprises:
performing global gray-level thresholding to obtain a first initial contour of said anatomic feature; and
performing local gray-level thresholding using said first initial contour to obtain said first feature contour.

### Claim 5 (depends on 1)

A method as recited in claim 1, wherein performing iterative gray-level thresholding comprises:
obtaining a global gray-level histogram of said first image including said anatomic feature;
obtaining a plurality of global gray-level threshold values using said global gray-level histogram;
generating a binary image using each of said plurality of global gray-level threshold values; and
determining a first initial contour using said binary images.

### Claim 6 (depends on 1)

A method as recited in claim 1, comprising:
obtaining a radiographic chest image as said first image;
digitizing said radiographic chest image to obtain a digitized chest image; and
detecting lung contours in said digitized chest image.

### Claim 7 (depends on 6)

A method as recited in claim 6, comprising:
obtaining a global gray-level histogram of a selected portion of said digitized chest image;
identifying lung and mediastinum peaks in said global gray-level histogram;
identifying a minimum in said global gray-level histogram between said lung and mediastinum peaks; and
selecting a plurality of gray-level threshold values between said minimum and said lung peak.

### Claim 8 (depends on 7)

A method as recited in claim 7, further comprising:
generating a plurality of binary images using said digitized chest image at each of said plurality of gray-level threshold values, respectively;
producing a plurality of contours of each of said plurality of binary images, respectively;
determining a plurality of centroids for each of said plurality of contours, respectively;
determining a centroid limit in said digitized chest image; and
selecting pixels located within each of said plurality of contours when a corresponding one of said centroids is located within said centroid limit.

### Claim 9 (depends on 1)

A method as recited in claim 1, comprising:
obtaining a radionuclide chest image as said second image;
performing noise analysis on said radionuclide chest image;
selecting a threshold value based on said noise analysis;
generating a binary image using said threshold value;
performing a morphological open operation on said binary image; and
detecting said second feature contour using said binary image after performing said morphological open operation.

### Claim 10 (depends on 1)

A method as recited in claim 1, comprising:
obtaining a radiographic hand image as said first image; and
performing iterative gray-level thresholding on said radiographic hand image.

### Claim 11 (depends on 10)

A method as recited in claim 10, comprising:
obtaining a radionuclide hand image as said second image;
performing noise analysis on said radionuclide hand image;
selecting a threshold value based on said noise analysis;
generating a binary image using said threshold value;
performing a morphological open operation on said binary image; and
detecting said second feature contour using said binary image after performing said morphological open operation.

### Claim 12 (depends on 1)

A method as recited in claim 1, comprising:
obtaining a radiographic image as said first image;
digitizing said radiographic image to obtain a digitized image;
detecting said first feature contour in said digitized image;
obtaining a radionuclide image as said second image;
performing noise analysis on said radionuclide image;
selecting a threshold value based on said noise analysis;
generating a binary image using said threshold value;
performing a morphological open operation on said binary image; and
detecting said second feature contour using said binary image after performing said morphological open operation.

### Claim 13 (depends on 12)

A method as recited in claim 12, wherein detecting said first feature contour in said digitized image comprises:
obtaining a first gray-level histogram of said radiographic image including said anatomic feature;
obtaining a plurality of first gray-level threshold values using said first gray-level histogram;
generating a first binary image of said region at each of said plurality of first gray-level threshold values;
determining a first initial contour using said first binary images;
placing a plurality of regions of interest (ROIs) on said first initial contour;
producing a respective plurality of second gray-level histograms of pixels located within each of said plurality of ROIs;
determining a respective plurality of second gray-level threshold values using said plurality of second gray-level histograms;
selecting pixels within said ROIs using respective ones of said second gray-level thresholds;
generating a second binary image using said selected pixels; and
detecting a boundary of said second binary image as said first feature contour.

### Claim 14

A method of correlating images from two modalities, comprising:
obtaining a first image containing an anatomic feature using a first modality;
obtaining a second image containing said anatomic feature using a second modality;
detecting a first feature contour of said anatomic feature in said first image using iterative gray-level thresholding;
detecting a second feature contour of said anatomic feature in said second image: and
superposing said first feature contour and said second feature contour;
wherein performing iterative gray-level thresholding comprises obtaining a global gray-level histogram of said first image including said anatomic feature;
obtaining a first derivative of said global gray-level histogram to identify a peak and a minimum in said histogram;
determining gray-level values of said peak and minimum; and
selecting a predetermined number of gray-level values spaced between gray-level values of said peak and said minimum as said global gray-level threshold values.

### Claim 15

A method of correlating images from two modalities, comprising:
obtaining a first image containing an anatomic feature using a first modality;
obtaining a second image containing said anatomic feature using a second modality;
detecting a first feature contour of said anatomic feature in said first image using iterative gray-level thresholding;
detecting a second feature contour of said anatomic feature in said second image; and
superposing said first feature contour and said second feature contour;
wherein performing iterative gray-level thresholding comprises;
obtaining a global gray-level histogram of said first image including said anatomic feature;
obtaining a plurality of global gray-level threshold values using said global gray-level histogram;
generating a binary image using each of said plurality of global gray-level threshold values; and
determining a first initial contour using said binary images; and
wherein determining said first initial contour comprises:
selecting pixels in said first image having a gray-level value greater than a predetermined gray-level threshold value and less than a corresponding one of said global gray-level threshold values to obtain at least one region;
producing a boundary of said at least one region;
determining whether said at least one region is located within said anatomic feature;
eliminating a region determined not to be in said anatomic feature; and
repeating said steps of selecting pixels, producing a boundary, determining whether said at least one region is located within said anatomic feature and eliminating a region for each of remaining ones of said global gray-level threshold values.

### Claim 16 (depends on 15)

A method as recited in claim 15, wherein said steps of determining whether said at least one region is located within said anatomic feature and eliminating a region comprise:
determining a centroid limit for said anatomic feature;
determining a centroid of said at least one region; and
eliminating said at least one region if said centroid is located outside said centroid limit.

### Claim 17 (depends on 16)

A method as recited in claim 16, comprising:
selecting a first centroid limit for use with selected ones of said global gray-level thresholds; and
selecting a second centroid limit for use with remaining ones of said global gray-level thresholds.

### Claim 18 (depends on 15)

A method as recited in claim 15, further comprising:
performing a morphological open operation on a selected one of said binary images.

### Claim 19

A method as of correlating images from two modalities, comprising:
obtaining a first image containing an anatomic feature using a first modality;
obtaining a second image containing said anatomic feature using a second modality;
detecting a first feature contour of said anatomic feature in said first image using iterative gray-level thresholding;
detecting a second feature contour of said anatomic feature in said second image; and
superposing said first feature contour and said second feature contour;
wherein performing iterative gray-level thresholding comprises;
performing global gray-level thresholding to obtain a first initial contour of said anatomic feature; and
performing local gray-level thresholding using said first initial contour to obtain said first feature contour; and
wherein performing local gray-level thresholding comprises:
placing a plurality of regions of interest (ROIs) on said first initial contour;
producing a respective plurality of gray-level histograms of pixels located within each of said plurality of ROIs;
determining a respective plurality of gray-level threshold values using said plurality of gray-level histograms;
selecting pixels within said ROIs using respective ones of said gray-level thresholds;
generating a binary image using said selected pixels; and
detecting a boundary of said binary image as said first feature contour.

### Claim 20 (depends on 19)

A method as recited in claim 19, comprising placing said plurality of ROIs on said first initial contour such that said ROIs overlap each other.

### Claim 21 (depends on 19)

A method as recited in claim 19, wherein determining said gray-level threshold comprises:
detecting at least one minimum in said gray-level histogram;
determining a gray-level value for each of said at least one minimum; and
selecting a gray-level threshold value as a largest one of said gray-level value for each of said at least one minimum.

### Claim 22 (depends on 19)

A method as recited in claim 19, further comprising performing a morphological open operation on said binary image.

### Claim 23

A method of correlating images from two modalities, comprising:
obtaining a radiographic chest image containing lung having a plurality of pixels as a first image using a first modality;
obtaining a second image containing said lung using a second modality:
detecting an initial lung contour in said first image using iterative gray-level thresholding:
placing a plurality of regions of interest (ROIs) on said initial lung contour;
selecting a gray-level threshold value in each of said ROIs;
selecting pixels in said ROIs based upon corresponding ones of said gray-level threshold values;
detecting a second lung contour in said second image; and
superposing said first and second images.

### Claim 24 (depends on 23)

A method as recited in claim 23, comprising:
determining whether an ROI is located on at least one of a medial and a diaphragmatic aspect of said initial lung contour; and
selecting a mean value of pixels located within an ROI as said gray-level threshold for each ROI determined to be located on said at least one of said medial and said diaphragmatic aspects of said initial lung contour.

### Claim 25 (depends on 23)

A method as recited in claim 23, comprising:
determining whether an ROI is located on a lateral aspect of said initial lung contour;
generating a gray-level histogram for each ROI determined to be located on said lateral aspect of said initial lung contour;
determining at least one minimum in each of said pray-level histograms;
determining a gray level value for each of said at least one minimum in each of said gray-level histograms:
selecting an initial gray-level threshold value as a largest one of said gray-level value for each of said at least one minimum for each said ROI determined to be located on said lateral aspect of said initial lung contour; and
selecting a gray-level threshold value for each said ROI determined to be located on said lateral aspect of said initial lung contour as an average of a corresponding one of said initial gray-level threshold values and initial gray-level threshold values of a neighboring two ROIs of said ROI determined to be located on said lateral aspect of said initial lung contour.

### Claim 26 (depends on 23)

A method as recited in claim 23, comprising:
placing an additional first ROI over the costophrenic angle; and
placing an additional second ROI over the cardiophrenic angle.

### Claim 27

A method of correlating images from two modalities, comprising:
obtaining a first image containing an anatomic feature using a first modality;
obtaining a second image containing said anatomic feature using a second modality;
detecting a first feature contour of said anatomic feature in said first image using iterative gray-level thresholding;
detecting a second feature contour of said anatomic feature in said second image; and superposing said first feature contour and said second feature contour;
wherein said superposing step comprises:
determining a first ratio of a first maximum width of said first feature contour to a second maximum width of said second feature contour;
determining a second ratio of a first maximum height of said first feature contour to a second maximum height of said second feature contour;
using said first and second ratios as first and second direction scaling factors, respectively; and
scaling said first feature contour using said first and second ratios.

### Claim 28 (depends on 27)

A method as recited in claim 27, further comprising:
performing horizontal profile analysis on said first and second images to identify a predetermined location in each of said first and second images;
shifting said second image relative to said first image in said first and second directions to align and overlay said first and second images using said predetermined location in each of said first and second images; and
displaying said first image aligned and overlaid with said second image.

### Claim 29 (depends on 28)

A method as recited in claim 28, further comprising:
displaying said first and second feature contours on said first image aligned and overlaid with said second image.

### Claim 30

A method as of correlating images from two modalities, comprising:
obtaining a first image containing an anatomic feature using a first modality;
obtaining a second image containing said anatomic feature using a second modality;
detecting a first feature contour of said anatomic feature in said first image using iterative gray-level thresholding;
detecting a second feature contour of said anatomic feature in said second image;
superposing said first feature contour and said second feature contour;
obtaining a chest image as said first and second images;
obtaining first and second lung contours as said first and second feature contours;
determining a first ratio of a first maximum width of said first lung contour to a second maximum width of said second lung contour;
determining a second ratio of a first maximum height of said first lung contour to a second maximum height of said second lung contour;
using said first and second ratios as first and second direction scaling factors, respectively; and
scaling said first feature contour using said first and second ratios.

### Claim 31 (depends on 30)

A method as recited in claim 30, further comprising:
performing horizontal profile analysis on said first and second images to identify lung apices in each of said first and second images;
shifting said second image relative to said first image in said first and second directions to align and overlay said first and second images using said lung apices in each of said first and second images; and
displaying said first image aligned and overlaid with said second image.

### Claim 32

A method of correlating images from two modalities, comprising:
obtaining a first image containing an anatomic feature using a first modality;
obtaining a second image containing said anatomic feature using a second modality;
detecting a first feature contour of said anatomic feature in said first image using iterative gray-level thresholding;
detecting a second feature contour of said anatomic feature in said second image;
superposing said first feature contour and said second feature contour;
obtaining a radiographic image as said first image; and
performing iterative gray-level thresholding on said radiographic image;
wherein performing iterative gray-level thresholding comprises:
placing a plurality of ROIs on said radiographic image;
generating a plurality of gray-level histograms for each of said ROIs, respectively;
determining a plurality of gray-level threshold values for each of said ROIs, respectively, using corresponding ones of said gray-level histograms; and
modifying a gray-level value of a predetermined pixel in selected ones of said ROIs using respective ones of said gray-level threshold values.

### Claim 33

A system for correlating images from two different modalities, comprising:
a first image acquisition device;
a second image acquisition device;
a first contouring circuit to provide first contours of a feature in an image obtained using said first image acquisition device by iteratively using estimated contours derived from a plurality of successive gray-level thresholding operations at a respective plurality of gray-level thresholds, including thresholding said first image using one of said plurality of gray-level thresholds, determining whether to alter pixels in said first image, and thresholding said first image using a subsequent one of said gray-level thresholds after said determining step;
a second contouring circuit, connected to said second image acquisition device, to provide second contours of said feature in said second image;
a superposition circuit connected to said first and second contouring circuits; and
a display.

### Claim 34 (depends on 33)

A system as recited in claim 33, wherein:
said first image acquisition device comprises a radiographic image acquisition device; and
said second image acquisition device comprises a radionuclide image acquisition device.

### Claim 35 (depends on 33)

A system as recited in claim 33, comprising:
first and second image memories connected to said first and second image acquisition devices, respectively.

### Claim 36 (depends on 33)

A system as recited in claim 33, wherein said superposition circuit comprises:
a scaling circuit;
a profile analysis circuit;
an image overlaying circuit; and
a circuit for overlaying said contours on an overlayed image comprised of said first and second images being overlayed.

### Claim 37 (depends on 33)

A system as recited in claim 33, wherein said first contouring circuit comprises:
a local gray-level thresholding circuit; and
a global gray-level thresholding circuit.

### Claim 38

A method of determining an image contour, comprising:
obtaining a first image containing an anatomic feature;
obtaining a plurality of gray-level thresholds in said first image;
thresholding said first image using one of said plurality of gray-level thresholds;
determining whether to alter pixels in said first image based upon said thresholding;
thresholding said first image using a second one of said plurality of gray-level thresholds after said determining step;
generating a plurality of initial contours using said thresholding steps; and
detecting a first feature contour of said anatomic feature using said plurality of initial contours.

### Claim 39 (depends on 38)

A method as recited in claim 38, comprising:
determining a first gray-level threshold used to produce a first binary image; and
increasing a gray-level value of each subsequent gray-level threshold used to produce respective subsequent binary images.

### Claim 40 (depends on 39)

A method as recited in claim 39, wherein determining said first gray-level threshold comprises:
generating a gray-level histogram of said first image; and
obtaining a first derivative of said histogram;
selecting a gray-level value corresponding to a minimum in said histogram.

### Claim 41 (depends on 40)

A method as recited in claim 40, comprising:
selecting a gray-level value corresponding to a maximum in said histogram; and
obtaining said gray-level thresholds between said gray-level values corresponding to said minimum and said maximum.

### Claim 42 (depends on 38)

A method as recited in claim 38, comprising:
thresholding said image using a smallest one of said plurality of thresholds;
producing a corresponding binary image;
processing said binary image; and
repeating said thresholding, producing and processing steps using successively higher ones of said plurality of thresholds.

### Claim 43 (depends on 42)

A method as recited in claim 42, wherein processing said binary image comprises:
eliminating portions of said binary image not meeting predetermined criteria.

### Claim 44 (depends on 38)

A method as recited in claim 38, comprising:
altering pixels in said first image; and
thresholding said first image after altering said pixels using said second one of said plurality of gray-level thresholds.

### Claim 45 (depends on 44)

A method as recited in claim 44, wherein:
altering said pixels comprises setting said pixels to a predetermined gray-level value.

### Claim 46 (depends on 38)

A method as recited in claim 38, comprising:
performing a respective plurality of thresholding operations using said plurality of gray-level thresholds; and
determining whether to alter pixels after each thresholding operation;
wherein a second and subsequent ones of said plurality of thresholding operations are performed after performing a respective determining step.

### Claim 47 (depends on 46)

A method as recited in claim 46, comprising:
if a pixel is determined to be altered, setting said pixel to a gray-level value below that of a gray-level threshold used in a subsequent thresholding operation.

### Claim 48 (depends on 38)

A method as recited in claim 38, wherein said determining step comprises:
determining said pixels to be in a region not a part of said anatomic feature; and
eliminating said pixels from consideration in a subsequent thresholding operation.

### Claim 49

A method of determining a feature contour, comprising:
obtaining a digitized chest image containing at least one lung;
obtaining a global gray-level histogram of a selected portion of said digitized chest image;
identifying lung and mediastinum peaks in said global gray-level histogram;
identifying a minimum in said global gray-level histogram between said lung and mediastinum peaks;
selecting a plurality of gray-level threshold values between said minimum and said lung peak; and
detecting a feature contour of said at least one lung in said chest image by iteratively using said plurality of gray-level threshold values.

### Claim 50 (depends on 49)

A method as recited in claim 49, comprising:
obtaining said digitized chest image using a first modality;
obtaining a second image containing said at least one lung using a second modality; and
superposing said digitized image and said second image.

### Claim 51 (depends on 49)

A method as recited in claim 49, wherein iteratively using said plurality of gray-level thresholds comprises:
performing global gray-level thresholding to obtain an initial contour of said lung; and
performing local gray-level thresholding using said initial contour to obtain said feature contour.

### Claim 52 (depends on 51)

A method as recited in claim 51, wherein performing local gray-level thresholding comprises:
placing a plurality of regions of interest (ROIs) on said initial contour;
producing a respective plurality of gray-level histograms of pixels located within each of said plurality of ROIs;
determining a respective plurality of gray-level threshold values using said plurality of gray-level histograms;
selecting pixels within said ROIs using respective ones of said gray-level thresholds;
generating a binary image using said selected pixels; and
detecting a boundary of said binary image as said feature contour.
