# US 12,004,849 B2 · Systems, methods, and computer-readable media for non-rigid registration of electromagnetic navigation space to CT volume

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,004,849 B2
- Document ID: US12004849B2
- Status: Granted
- Kind: B2
- Application number: 16/211,314
- Filed: 2018-12-06
- Published: 2019-06-13
- Granted: 2024-06-11
- Earliest claimed: 2017-12-11
- Assignees: Covidien LP
- Applicants: Covidien LP
- Inventors: Lev A. Koyrakh; Ron Barak; Oren P. Weingarten; Alexander Nepomniashchy
- Web page: https://patentgrep.com/patent/US12004849B2

## CPC

- A61B 5/066: {Superposing sensor position on an image of the patient, e.g. obtained by ultrasound or x-ray imaging}
- A61B 1/2676: {Bronchoscopes}
- A61B 5/062: {using magnetic field}
- A61B 5/7253: {characterised by using transforms}
- A61B 6/032: {Transmission computed tomography [CT]}
- A61B 34/20: Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
- G06T 7/30: Determination of transform parameters for the alignment of images, i.e. image registration
- G06T 7/33: using feature-based methods
- A61B 6/504: {for diagnosis of blood vessels, e.g. by angiography}
- A61B 6/5247: {combining images from an ionising-radiation diagnostic technique and a non-ionising radiation diagnostic technique, e.g. X-ray and ultrasound}
- A61B 6/547: {involving tracking of position of the device or parts of the device}
- A61B 2017/00809: {Lung operations}
- A61B 2034/105: {Modelling of the patient, e.g. for ligaments or bones}
- A61B 2034/2051: {Electromagnetic tracking systems}
- A61B 2034/2072: {Reference field transducer attached to an instrument or patient}
- A61B 2090/363: {Use of fiducial points}
- A61B 2090/364: {Correlation of different images or relation of image positions in respect to the body}
- A61B 2090/3925: {ultrasonic}
- G06T 2207/10081: Computed x-ray tomography [CT]
- G06T 2207/30061: Lung
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Methods, systems, and computer-readable media for registering electromagnetic navigation data of a luminal network to a 3D model of the luminal network includes accessing a 3D model of a luminal network based on computed tomographic (CT) images of the luminal network, the 3D model corresponding to a CT coordinate space, selecting a plurality of reference points within the 3D model of the luminal network, accessing a plurality of survey points within the luminal network, the plurality of survey points being based on electromagnetic navigation data and corresponding to a body coordinate space, correlating the plurality of reference points and the plurality of survey points to determine pairs of correlated reference points and survey points; and deriving a transformation that maps the body coordinate space to the CT coordinate space based on the pairs of correlated reference points and survey points.

## Claims

### Claim 1

A system comprising:
a location sensor capable of being navigated within a luminal network inside a patient's body;
an electromagnetic field generator configured to detect a location of the location sensor as it is navigated within the luminal network;
a computing device including a processor and a memory; and
an application, stored in the memory and executed by the processor, which when executed, causes the computing device to:
during a planning phase:
receive a 3D model of the luminal network, the 3D model being generated from computed tomographic (CT) images of the luminal network, the 3D model corresponding to a CT coordinate space;
generate a navigation plan; and
select, at predetermined distance intervals, a plurality of reference points within the 3D model of the luminal network, the reference points being along a center of the luminal network of the 3D model;
during a registration phase:
receive a plurality of survey points within the luminal network, the plurality of survey points corresponding to a body coordinate space and being based on locations of the location sensor while the location sensor is navigated within the luminal network;
correlate the plurality of survey points with the plurality of reference points to determine pairs of correlated reference points and survey points; and
derive a thin plate splines (TPS) transformation that maps the body coordinate space to the CT coordinate space based on the pairs of correlated reference points and survey points by interpolating additional points in the body coordinate space and the CT coordinate space based on an inverse multi-rigid registration, yielding a derived TPS transformation;
during a navigation phase different from the registration phase:
receive a further survey point from the location sensor, which is navigated according to the navigation plan;
determine a location of the location sensor in the CT coordinate space by determining a point in the 3D model corresponding to the further survey point by applying the derived TPS transformation to the further survey point; and
update the derived TPS transformation based on a plurality of second additional correlated points interpolated from the inverse multi-rigid registration performed on the further survey point while navigation is in progress.

### Claim 2 (depends on 1)

The system of claim 1, wherein the luminal network is an airway of a patient.

### Claim 3 (depends on 2)

The system of claim 2, wherein the 3D model is a model of the airway of the patient.

### Claim 4 (depends on 1)

The system of claim 1, wherein the application stored in the memory when executed, further causes the computing device to:
select fiducial points from the plurality of survey points and the plurality of reference points, wherein the fiducial points are at intersections in the luminal network where the luminal network branches apart.

### Claim 5 (depends on 4)

The system of claim 4, wherein in correlating the plurality of survey points with the plurality of reference points, the application stored in the memory when executed, further causes the computing device to:
match the fiducial points selected from the plurality of survey points with the fiducial points selected from the plurality of reference points, yielding matched fiducial points, wherein the pairs of correlated reference points and survey points include the matched fiducial points.

### Claim 6 (depends on 1)

The system of claim 1, wherein correlating the plurality of survey points with the plurality of reference points is based on an inverse registration of the CT coordinate space to the body coordinate space.

### Claim 7 (depends on 6)

The system of claim 6, wherein the inverse registration is at least one of: an inverse of an optimized zone registration or an inverse of a multi-rigid registration.

### Claim 8 (depends on 1)

The system of claim 1, wherein the application stored in the memory, when executed, further causes the computing device to update the derived TPS transformation during the navigation phase by:
receiving a plurality of further survey points;
determining a registration of the body coordinate space to the CT coordinate space based on the plurality of further survey points;
matching the plurality of further survey points with the plurality of reference points based on the registration; and
updating the derived TPS transformation based on the matching.

### Claim 9

A system comprising:
a computing device including a processor and a memory; and
an application, stored in the memory and executed by the processor, which when executed, causes the computing device to:
during a planning phase:
receive a 3D model of a luminal network inside a patient's body, the 3D model being generated from computed tomographic (CT) images of the luminal network and corresponding to a CT coordinate space;
generate a navigation plan; and
select, at predetermined distance intervals, a plurality of reference points within the 3D model of the luminal network, the reference points being along a center of the luminal network of the 3D model;
during a registration phase:
receive a plurality of survey points within the luminal network, the plurality of survey points corresponding to a body coordinate space and being based on locations of a location sensor while the location sensor is navigated within the luminal network;
correlate the plurality of reference points and the plurality of survey points to determine a plurality of correlated points;
perform an inverse multi-rigid registration of the CT coordinate space to the body coordinate space based on the plurality of correlated points;
interpolate a plurality of additional correlated points based on the inverse multi-rigid registration; and
derive a TPS transformation that maps the body coordinate space to the CT coordinate space from the plurality of correlated points and the plurality of additional correlated points, yielding a derived TPS transformation; and
during a navigation phase different from the registration phase:
receive a further survey point from the location sensor, which is navigated according to the navigation plan;
determine a location of the location sensor in the CT coordinate space by determining a point in the 3D model corresponding to the further survey point by applying the derived TPS transformation to the further survey point; and
update the derived TPS transformation based on a plurality of second additional correlated points interpolated from the inverse multi-rigid registration performed on the further survey point while navigation is in progress.

### Claim 10

A system comprising:
a computing device including a processor and a memory; and
an application, stored in the memory and executed by the processor, which when executed, causes the computing device to:
during a planning phase:
receive a 3D model of a luminal network inside a patient's body, the 3D model being generated from computed tomographic (CT) images of the luminal network and corresponding to a CT coordinate space;
generate a navigation plan; and
select, at predetermined distance intervals, a plurality of reference points within the 3D model of the luminal network, the reference points being along a center of the luminal network of the 3D model;
during a registration phase:
receive a plurality of survey points within the luminal network, the plurality of survey points being based on electromagnetic navigation data and corresponding to a body coordinate space;
perform an inverse multi-rigid registration of the CT coordinate space to the body coordinate space based on the plurality of reference points and the plurality of survey points; and
derive a thin plate splines (TPS) transformation that maps the body coordinate space to the CT coordinate space from a plurality of additional points interpolated based on the inverse multi-rigid registration, yielding a derived TPS transformation; and
during a navigation phase different from the registration phase:
receive a further survey point from a location sensor, which is navigated according to the navigation plan;
determine a location of the location sensor in the CT coordinate space by determining a point in the 3D model corresponding to the further survey point by applying the derived TPS transformation to the further survey point; and
update the derived TPS transformation based on a plurality of second additional correlated points interpolated from the inverse multi-rigid registration performed on the further survey point while navigation is in progress.

### Claim 11 (depends on 10)

The system of claim 10, wherein the luminal network is an airway of a patient.

### Claim 12 (depends on 11)

The system of claim 11, wherein the 3D model is a model of the airway of the patient.

### Claim 13 (depends on 10)

The system of claim 10, further including selecting fiducial points from the plurality of survey points and the plurality of reference points wherein the fiducial points are at intersections in the luminal network where the luminal network branches apart.

### Claim 14 (depends on 13)

The system of claim 13, wherein performing the inverse registration includes performing the inverse registration based on the fiducial points.

### Claim 15 (depends on 10)

The system of claim 10, wherein performing the inverse registration of the CT coordinate space to the body coordinate space includes correlating the plurality of reference points and the plurality of survey points.

### Claim 16 (depends on 10)

The system of claim 10, wherein the inverse registration is at least one of: an inverse of an optimized zone registration or an inverse of a multi-rigid registration.

### Claim 17 (depends on 10)

The system of claim 10, wherein the application, when executed, further causes the computing device to update the derived TPS transformation during the navigation phase by:
receiving a plurality of further survey points;
determining a registration of the body coordinate space to the CT coordinate space based on the plurality of further survey points;
matching the plurality of further survey points with the plurality of reference points based on the registration; and
updating the derived TPS transformation based on the matching.

### Claim 18 (depends on 10)

The system of claim 10, further comprising:
a location sensor capable of being navigated within the luminal network inside the patient's body; and
an electromagnetic field generator configured to detect the location of the location sensor as it is navigated within the luminal network.