# US 10,864,043 B2 · Interactive placement of a 3D digital representation of a surgical device or anatomic feature into a 3D radiologic image for pre-operative planning

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,864,043 B2
- Document ID: US10864043B2
- Status: Granted
- Kind: B2
- Application number: 16/010,925
- Filed: 2018-06-18
- Published: 2019-08-15
- Granted: 2020-12-15
- Earliest claimed: 2018-02-09
- Applicants: Kathleen M Douglas; Robert E Douglas
- Inventors: Kathleen M Douglas; Robert E Douglas
- Web page: https://patentgrep.com/patent/US10864043B2

## CPC

- A61B 34/10: Computer-aided planning, simulation or modelling of surgical operations
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- A61B 2034/102: {Modelling of surgical devices, implants or prosthesis}
- A61B 2034/104: {Modelling the effect of the tool, e.g. the effect of an implanted prosthesis or for predicting the effect of ablation or burring}
- A61B 2034/105: {Modelling of the patient, e.g. for ligaments or bones}
- A61B 2034/108: {Computer aided selection or customisation of medical implants or cutting guides}
- A61B 2090/367: {creating a 3D dataset from 2D images using position information}
- A61F 2/82: Devices providing patency to, or preventing collapsing of, tubular structures of the body, e.g. stents (stent-grafts for tubular structures of the body other than blood vessels A61F2/04; stent-grafts for blood vessels A61F2/07)
- G06T 2219/004: Annotating, labelling
- G06T 2219/2004: Aligning objects, relative positioning of parts
- G06T 2219/2012: Colour editing, changing, or manipulating; Use of colour codes
- G06T 2219/2016: Rotation, translation, scaling
- G06T 2219/2021: Shape modification
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- A61F: FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS (dental prosthetics A61C)

## Abstract

An interactive placement of a digital representation of a surgical device or anatomic feature into a radiologic 3D medical image that contains a feature of interest is described. Integration of the digital representation of the surgical device or anatomic feature with the radiologic 3D medical image facilitates pre-operative surgical planning and surgical device selection. Location indicators, annotations, and registration markers may be overlaid on selected volume images.

## Claims

### Claim 1

A method comprising:
performing segmentation of a volume at a first time point to define a structure wherein the structure at the first time point has a first size, a first shape and contains a first salient feature;
performing filtering of the volume at the first time point wherein portions of the volume at the first time point not pertaining to the structure at the first time point are removed;
performing segmentation of a volume at a second time point to define a structure wherein the structure at the second time point has a second size, a second shape and contains a second salient feature wherein the second salient feature corresponds to the first salient feature of the structure at the first time point;
performing filtering of the volume at the second time point wherein portions of the volume at the second time point not pertaining to the structure at the second time point are removed;
superimposing the structure at the first time point with the structure at the second time point comprising:
responsive to a first user input, moving the first salient feature of the structure at the first time point relative to the second salient feature of the structure at the second time point such that the first salient feature of the structure at the first time point is superimposed on the second salient feature of the structure at the second time point; and
responsive to a second user input, aligning the first salient feature of the structure at the first time point relative to the second salient feature of the structure at the second time point such that the orientation of the structure at the first time point matches the orientation of the structure at the second time point; and
responsive to a third user input, displaying the structure at the first time point transparently and the structure at the second time point non-transparently in a superimposed fashion, wherein the structure at the first time point and the structure at the second time point relate to human body parts.

### Claim 2 (depends on 1)

The method of claim 1 wherein a user assesses interval changes of one of the group consisting of:
the first size of the structure at the first time point compared with the second size of the structure at the second time point; and
the first shape of the structure at the first time point compared with the second shape of the structure at the second time point.

### Claim 3 (depends on 1)

The method of claim 1 further comprising: moving the structure at the first time point such that the first salient feature of the structure at the first time point is not superimposed on the second salient feature of the structure at the second time point.

### Claim 4 (depends on 3)

The method of claim 3 further comprising: displaying the structure at the first time point adjacent to the structure at the second time point.

### Claim 5 (depends on 1)

The method of claim 1 further comprising replacing voxels from the structure at the first time point that overlap with voxels from the structure at the second time point.

### Claim 6 (depends on 1)

The method of claim 1 comprising displaying the structure at the first time point and the structure at the second time point on an augmented reality display.

### Claim 7 (depends on 1)

The method of claim 1 comprising displaying the structure at the first time point and the structure at the second time point on a virtual reality display.

### Claim 8 (depends on 1)

The method of claim 1 comprising displaying the structure at the first time point and the structure at the second time point on a monitor.

### Claim 9

An apparatus comprising:
an IO device; and
an image processor in communication with the IO device, the image processors comprising a program stored on computer-readable non-transitory media, the program comprising:
instructions that perform segmentation of a volume at a first time point to define a structure wherein the structure at the first time point has a first size, a first shape and contains a first salient feature;
instructions that perform filtering of the volume at the first time point wherein portions of the volume at the first time point not pertaining to the structure at the first time point are removed;
instructions that perform segmentation of a volume at a second time point to define a structure wherein the structure at the second time point has a second size, second shape and contains a second salient feature wherein the second salient feature corresponds to the first salient feature of the structure at the first time point;
instructions that perform filtering of the volume at the second time point wherein portions of the volume at the second time point not pertaining to the structure at the second time point are removed:
instructions that superimpose the structure at the first time point with the structure at the second time point comprising:
responsive to a first user input, moving the first salient feature of the structure at the first time point relative to the second salient feature of the structure at the second time point such that the first salient feature of the structure at the first time point is superimposed on the second salient feature of the structure at the second time point; and
responsive to a second user input, aligning the first salient feature of the structure at the first time point relative to the second salient feature of the structure at the second time point such that the orientation of the structure at the first time point matches the orientation of the structure at the second time point; and
instructions that responsive to a third user input, display the structure at the first time point transparently and the structure at the second time point non-transparently in a superimposed fashion, wherein the structure at the first time point and the structure at the second time point relate to human body parts.

### Claim 10 (depends on 9)

The apparatus of claim 9 wherein the instructions provide capability for a user to assess interval changes of one of the group consisting of:
the first size of the structure at the first time point compared with the second size of the structure at the second time point; and
the first shape of the structure at the first time point compared with the second shape of the structure at the second time point.

### Claim 11 (depends on 9)

The apparatus of claim 9 comprising instructions that move the structure at the first time point such that the first salient feature of the structure at the first time point is not superimposed on the second salient feature of the structure at the second time point.

### Claim 12 (depends on 11)

The apparatus of claim 11 comprising instructions that display the structure at the first time point adjacent to the structure at the second time point.

### Claim 13 (depends on 9)

The apparatus of claim 9 further comprising instructions that replace voxels from the structure at the first time point that overlap with voxels from the structure at the second time point.

### Claim 14 (depends on 9)

The apparatus of claim 9 comprising instructions that display the structure at the first time point and the structure at the second time point on an augmented reality display.

### Claim 15 (depends on 9)

The apparatus of claim 9 comprising displaying the structure at the first time point and the structure at the second time point on a virtual reality display.

### Claim 16 (depends on 9)

The apparatus of claim 9 comprising displaying the structure at the first time point and the structure at the second time point on a monitor.