# US 12,118,650 B2 · Systems and methods for rendering objects translucent in x-ray images

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,118,650 B2
- Document ID: US12118650B2
- Status: Granted
- Kind: B2
- Application number: 17/760,602
- Filed: 2020-09-16
- Published: 2022-10-27
- Granted: 2024-10-15
- Earliest claimed: 2019-09-16
- Assignees: NuVasive, Inc.
- Applicants: Nuvasive, Inc.
- Inventors: Sean O'Connor; Alexander Hsu; Kara Robinson
- Web page: https://patentgrep.com/patent/US12118650B2

## CPC

- G06T 15/20: Perspective computation
- A61B 90/37: {Surgical systems with images on a monitor during operation}
- G06T 5/77: Retouching; Inpainting; Scratch removal
- G06T 7/0012: {Biomedical image inspection}
- G06T 7/32: using correlation-based methods
- G06T 12/30: Image post-processing, e.g. metal artefact correction
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06V 20/60: Type of objects
- G16H 30/40: for processing medical images, e.g. editing
- G06T 2200/04: involving 3D image data
- G06T 2207/10048: Infrared image
- G06T 2207/10064: Fluorescence image
- G06T 2207/10072: Tomographic images
- G06T 2207/10116: X-ray image
- G06T 2207/10132: Ultrasound image
- G06T 2207/20212: Image combination
- G06T 2207/30004: Biomedical image processing
- G06T 2207/30012: Spine; Backbone
- G06T 2207/30052: Implant; Prosthesis
- G06T 2210/41: Medical
- G06T 2210/62: Semi-transparency
- G06T 2211/40: Computed tomography
- G06T 2211/448: involving metal artefacts, streaking artefacts, beam hardening or photon starvation
- G06V 2201/034: of medical instruments
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

The present disclosure includes systems, methods and media for rendering objects translucent and for recovery of anatomical information blocked by the objects in medical images.

## Claims

### Claim 1

A method for rendering metal objects translucent in medical images, the method comprising:
receiving, by a computer, a three-dimensional dataset of a subject;
determining, by the computer, an imaging space of an image capturing device, wherein the image capturing device is configured to generate a two-dimensional image of the subject from the imaging space;
acquiring, by the image capturing device, a two-dimensional image of the subject from the imaging space;
registering, by the computer, the three-dimensional dataset of the subject to the imaging space of the image capturing device, thereby generating a registered three-dimensional dataset; and
predicting, by the computer, at least part of the two-dimensional image of the subject using the registered three-dimensional dataset, wherein the at least part of the two-dimensional image of the subject includes one or more metal objects, and wherein the prediction of the at least part of the two-dimensional image of the subject includes a first set of anatomical information of the subject that is blocked by the one or more metal objects.

### Claim 2 (depends on 1)

The method of claim 1, further comprising receiving, by the computer, the two-dimensional image of the subject from the image capturing device.

### Claim 3 (depends on 2)

The method of claim 2, further comprising, subsequent to the step of predicting at least a part of the two-dimensional image of the subject, generating, by the computer, an improved two-dimensional image of the subject based on the two-dimensional image and the prediction of the at least part of the two-dimensional image of the subject.

### Claim 4 (depends on 3)

The method of claim 3, wherein the improved two-dimensional image is unobscured by the one or more metal objects, and wherein the improved two-dimensional image includes the first set of anatomical information of the subject.

### Claim 5 (depends on 3)

The method of claim 3, wherein generating the improved two-dimensional image of the subject comprises:
identifying, by the computer, a plurality of pixels that at least partly contains the one or more metal objects;
optionally modifying, by the computer, data of the plurality of pixels; and
combining, by the computer, the prediction of the at least part of the two-dimensional image of the subject and the data of the plurality of pixels pixel by pixel.

### Claim 6 (depends on 3)

The method of claim 3 further comprising, presenting, by a digital display, the improved two-dimensional image to a user during a medical procedure.

### Claim 7 (depends on 6)

The method of claim 6, wherein presenting the improved two-dimensional image comprises presenting the prediction of the first set of blocked anatomical information as an overlay superimposed on the two-dimensional image thereby facilitating the medical procedure.

### Claim 8 (depends on 2)

The method of claim 2, wherein the two-dimensional image is an X-ray image, and wherein the one or more metal objects block the first set of anatomical information of the subject therewithin.

### Claim 9 (depends on 8)

The method of claim 8 further comprising adjusting registration of the three-dimensional dataset of the subject to the imaging space of the image capturing device until prediction of a second set of unblocked anatomical information of the two-dimensional image satisfies a pre-determined criterion.

### Claim 10 (depends on 9)

The method of claim 9, wherein adjusting registration of the three-dimensional dataset of the subject to the imaging space of the image capturing device comprises calculating a similarity of the prediction of the second set of unblocked anatomical information to the second set of unblocked anatomical information in the two-dimensional image.

### Claim 11 (depends on 2)

The method of claim 2, wherein the two-dimensional image is generated using projection imaging, and wherein the one or more metal objects block the first set of anatomical information of the subject within the two-dimensional image.

### Claim 12 (depends on 1)

The method of claim 1, wherein the three-dimensional dataset includes a CT dataset, a Magnetic Resonance Imaging dataset, a Positron Emission Tomography (PET) dataset, or an Ultrasound dataset of the subject.

### Claim 13 (depends on 12)

The method of claim 12, wherein the three-dimensional dataset is acquired prior to a medical procedure of the subject that includes introduction of metal objects to the subject, or wherein the three-dimensional dataset is free of the one or more metal objects.

### Claim 14 (depends on 1)

The method of claim 1, wherein one or more of: the registering of the three-dimensional dataset of the subject to the imaging space of the image capturing device; the prediction of the at least part of the two-dimensional image of the subject using the registered three-dimensional dataset; and the presentation of the improved two-dimensional image to the user is in real-time.

### Claim 15 (depends on 14)

The method of claim 14, wherein the real-time includes a time duration of less than 1 second, 0.8 seconds, 0.6 seconds, 0.5 seconds, 0.4 seconds, 0.3 seconds, 0.2 seconds, 0.1 seconds, 0.08 seconds, 0.06 seconds, 0.05 seconds, 0.02 seconds, or 0.01 seconds.

### Claim 16 (depends on 1)

The method of claim 1, wherein registering the three-dimensional dataset of the subject to the imaging space of the image capturing device comprises registering a first coordinate system of the three-dimensional dataset to a second coordinate system of the imaging space.

### Claim 17 (depends on 1)

The method of claim 1, wherein the three-dimensional dataset is acquired with metal items within a field of view thereof, and wherein registering the three-dimensional dataset of the subject to the imaging space of the image capturing device comprises tracking the metal items using infrared light in the imaging space and registering the metal pins in the three-dimensional dataset to the metal pins in the imaging space.

### Claim 18

A computer-implemented system comprising:
an image capturing device;
a digital processing device comprising a processor, a memory, and an operating system configured to perform executable instructions, the digital processing device in digital communication with the image capturing device; and
a computer program stored in the memory including instructions executable by the processor of the digital processing device to create a metal object removal application comprising:
a software module configured to receive a three-dimensional dataset of a subject;
a software module configured to determine by the computer, an imaging space of an image capturing device, wherein the image capturing device is configured to generate a two-dimensional image obtained of the subject from the imaging space;
a software module configured to register the three-dimensional dataset of the subject to the imaging space of the image capturing device, thereby generating a registered three-dimensional dataset; and
a software module configured to predict at least part of the two-dimensional image of the subject using the registered three-dimensional dataset, wherein the at least part of the two-dimensional image of the subject includes one or more metal objects, and wherein the prediction of the at least part of the two-dimensional image of the subject includes a first set of anatomical information of the subject that is blocked by the one or more metal objects.

### Claim 19

A method for rendering objects translucent in medical images, the method comprising:
receiving, by a computer, a three-dimensional dataset of a subject;
determining, by the computer, an imaging space of an image capturing device, wherein the image capturing device is configured to generate a two-dimensional image of the subject from the imaging space;
registering, by the computer, the three-dimensional dataset of the subject to the imaging space of the image capturing device, thereby generating a registered three-dimensional dataset;
predicting, by the computer, at least part of the two-dimensional image of the subject using the registered three-dimensional dataset, wherein the at least part of the two-dimensional image of the subject includes one or more objects, and wherein the prediction of the at least part of the two-dimensional image of the subject includes a first set of anatomical information of the subject that is blocked by the one or more objects;
further comprising acquiring, by the image capturing device, a two-dimensional image of the subject from the imaging space; and receiving, by the computer, the two-dimensional image of the subject from the image capturing device;
further comprising, subsequent to the step of predicting at least a part of the two-dimensional image of the subject, generating, by the computer, an improved two-dimensional image of the subject based on the two-dimensional image and the prediction of the at least part of the two-dimensional image of the subject;
wherein the improved two-dimensional image is unobscured by the one or more objects, and wherein the improved two-dimensional image includes the first set of anatomical information of the subject; and
wherein the one or more objects comprise metal.

### Claim 20 (depends on 19)

The method of claim 19 is without introduction of a radiation dosage to the subject.