# US 10,872,449 B2 · System and method for constructing virtual radial ultrasound images from CT data and performing a surgical navigation procedure using virtual ultrasound images

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,872,449 B2
- Document ID: US10872449B2
- Status: Granted
- Kind: B2
- Application number: 16/383,888
- Filed: 2019-04-15
- Published: 2019-11-07
- Granted: 2020-12-22
- Earliest claimed: 2018-05-02
- Assignees: Covidien LP
- Applicants: Covidien LP
- Inventors: Nicolas J. Merlet; Oren P. Weingarten
- Web page: https://patentgrep.com/patent/US10872449B2

## CPC

- A61B 6/032: {Transmission computed tomography [CT]}
- A61B 6/50: specially adapted for specific body parts; specially adapted for specific clinical applications
- A61B 6/5217: {extracting a diagnostic or physiological parameter from medical diagnostic data}
- A61B 6/5223: {generating planar views from image data, e.g. extracting a coronal view from a 3D image}
- A61B 6/5247: {combining images from an ionising-radiation diagnostic technique and a non-ionising radiation diagnostic technique, e.g. X-ray and ultrasound}
- A61B 8/12: in body cavities or body tracts, e.g. by using catheters
- A61B 34/20: Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
- G06T 3/06: Topological mapping of higher dimensional structures onto lower dimensional surfaces
- G06T 5/70: Denoising; Smoothing
- G06T 11/60: Creating or editing images; Combining images with text
- G06T 15/06: Ray-tracing
- G16H 50/30: for calculating health indices; for individual health risk assessment
- A61B 2034/2051: {Electromagnetic tracking systems}
- G06T 2207/10081: Computed x-ray tomography [CT]
- G06T 2207/20221: Image fusion; Image merging
- G06T 2207/30096: Tumor; Lesion
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

A system and method for constructing virtual radial ultrasound images from CT data includes receiving CT data, calculating a gradient of the CT data, and obtaining 2D slices by interpolation in 3D. The method further includes calculating transmission and reflection values on each radial line away from a center of each 2D slice to create a lesion image, converting the lesion image to polar, generating random noise in the polar image, and adding random comet tails to the polar image. A background image is created by computing the mean profile along the radius from a series of polar lesion images. The background image is merged with the polar lesion image and random noise is generated in the background of the merged image. Density variations are simulated in the merged image and coordinates of the merged image are transformed to Cartesian to construct a virtual radial ultrasound image.

## Claims

### Claim 1

A method for constructing virtual radial ultrasonic images from CT data, the method comprising:
receiving preoperative CT data of a branched luminal network;
calculating a gradient of the CT data;
obtaining 2D slices of the CT data by interpolation in 3D;
calculating transmission and reflection values on radial lines away from a center of each 2D slice to create a lesion image;
converting the lesion image to a polar lesion image;
generating random noise in the polar lesion image;
adding random comet tails to the polar lesion image based on the random noise;
creating a background image by computing a mean profile along a radius from a series of polar lesion images;
merging the background image with the polar lesion image to create a merged image;
generating random noise in a background of the merged image;
simulating density variations in the merged image; and
transforming coordinates of the merged image to Cartesian to construct a virtual radial ultrasound image.

### Claim 2 (depends on 1)

The method according to claim 1, wherein calculating transmission and reflection values on each radial line away from a center of each 2D slice to create a lesion image includes applying the formulas:
imVirt(ptCur)=|Rad·Grad|·|Grad|/(2·Vol){circumflex over ( )}2·valTransmit; and
imTransmit(ptCur)=(1−(|Grad|/(2·Vol)){circumflex over ( )}2.

### Claim 3 (depends on 1)

The method according to claim 1, further comprising copying a kernel in the polar lesion image using a model kernel representing what is seen in an isolated reflection.

### Claim 4 (depends on 3)

The method according to claim 3, wherein copying a kernel in the polar lesion image using a model kernel representing what is seen in an isolated reflection includes applying the formula:
imLesion(neiY,neiX)=max(imLesion(neiY,neiX),kernel*imReflection(peakY,peakX))
where neiY and neiX are lists of coordinates in a neighborhood of peakY and peakX.

### Claim 5 (depends on 1)

The method according to claim 1, further comprising simulating blurring of real ultrasound images around high density points in the polar lesion image.

### Claim 6 (depends on 1)

The method according to claim 1, wherein simulating blurring of real ultrasound images around high density points in the polar lesion image includes performing a plurality of grey linear closings for a plurality of directions between 0 and 180 degrees.

### Claim 7 (depends on 1)

The method according to claim 1, wherein adding random comet tails to the polar lesion image based on the random noise includes adding a comet to a peak by considering each neighbor of the peak (neiPeak), above and below, in a kernel mask.

### Claim 8 (depends on 7)

The method according to claim 7, further comprising copying to the right of each neighbor of the peak along a horizontal line a decreasing profile defined for each pixel as:
val(neiPeak)*ratioX(radiusDif)*ratioY(angleDif),
where radiusDif is a radius difference to the peak and angleDif is an angle difference to the peak for each pixel along a profile.

### Claim 9 (depends on 1)

The method according to claim 1, wherein merging the background image with the polar lesion image to create a merged image includes applying a heuristic formula.

### Claim 10 (depends on 9)

The method according to claim 9, wherein the heuristic formula is:
imBackground(y,x)=imBackground(y,x)+log(imBackground(y,x))*imLesion(y,x)*K.

### Claim 11

A system for constructing virtual radial ultrasound images from CT data, comprising a computing device configured to:
receive preoperative CT data of a branched luminal network;
calculate a gradient of the CT data;
obtain 2D slices of the CT data by interpolation in 3D;
calculate transmission and reflection values on radial lines away from a center of each 2D slice to create a lesion image;
convert the lesion image to a polar lesion image;
generate random noise in the polar lesion image;
add random comet tails to the polar lesion image based on the random noise;
create a background image by computing a mean profile along a radius from a series of polar lesion images;
merge the background image with the polar lesion image to create a merged image;
generate random noise in a background of the merged image;
simulate density variations in the merged image; and
transform coordinates of the merged image to Cartesian to construct a virtual radial ultrasound image.

### Claim 12 (depends on 11)

The system according to claim 11, wherein the computing device is configured to calculate transmission and reflection values on each radial line away from a center of each 2D slice to create a lesion image by applying the formulas:
imVirt(ptCur)=|Rad·Grad|·|Grad|/(2·Vol){circumflex over ( )}2·valTransmit; and
imTransmit(ptCur)=(1−(|Grad|/(2·Vol)){circumflex over ( )}2.

### Claim 13 (depends on 11)

The system according to claim 11, wherein the computing device is further configured to copy a kernel in the polar lesion image using a model kernel representing what is seen in an isolated reflection.

### Claim 14 (depends on 13)

The system according to claim 13, wherein the computing device is configured to copy a kernel in the polar lesion image using a model kernel representing what is seen in an isolated reflection by applying the formula:
imLesion(neiY,neiX)=max(imLesion(neiY,neiX),kernel*imReflection(peakY,peakX))
where neiY and neiX are lists of coordinates in a neighborhood of peakY and peakX.

### Claim 15 (depends on 11)

The system according to claim 11, wherein the computing device is further configured to simulate blurring of real ultrasound images around high density points in the polar lesion image.

### Claim 16 (depends on 11)

The system according to claim 11, wherein the computing device is configured to simulate blurring of real ultrasound images around high density points in the polar lesion image by performing a plurality of grey linear closings for a plurality of directions between 0 and 180 degrees.

### Claim 17 (depends on 11)

The system according to claim 11, wherein the computing device is configured to add random comet tails to the polar lesion image based on the random noise by adding a comet to a peak by considering each neighbor of the peak (neiPeak), above and below, in a kernel mask.

### Claim 18 (depends on 17)

The system according to claim 17, wherein the computing device is further configured to copy to the right of each neighbor of the peak along a horizontal line a decreasing profile defined for each pixel as:
val(neiPeak)*ratioX(radiusDif)*ratioY(angleDif),
where radiusDif is a radius difference to the peak and angleDif is an angle difference to the peak for each pixel along a profile.

### Claim 19 (depends on 11)

The system according to claim 11, wherein the computing device is configured to merge the background image with the polar lesion image to create a merged image by applying a heuristic formula.

### Claim 20 (depends on 19)

The system according to claim 19, wherein the heuristic formula is:
imBackground(y,x)=imBackground(y,x)+log(imBackground(y,x))*imLesion(y,x)*K.