# US 10,307,079 B2 · Method and system for surgical instrument guidance and tracking with position and orientation correction

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,307,079 B2
- Document ID: US10307079B2
- Status: Granted
- Kind: B2
- Application number: 14/603,959
- Filed: 2015-01-23
- Published: 2015-07-30
- Granted: 2019-06-04
- Earliest claimed: 2014-01-27
- Assignees: Edda Technology, Inc.
- Applicants: Edda Technology, Inc.
- Inventors: Guo-Qing Wei; Xin Dou; Li Fan; Xiaolan Zeng; Jian-Zhong Qian; Cheng-Chung Liang
- Web page: https://patentgrep.com/patent/US10307079B2

## CPC

- A61B 5/066: {Superposing sensor position on an image of the patient, e.g. obtained by ultrasound or x-ray imaging}
- A61B 6/12: Arrangements for detecting or locating foreign bodies
- A61B 6/463: {characterised by displaying multiple images or images and diagnostic data on one display}
- A61B 6/50: specially adapted for specific body parts; specially adapted for specific clinical applications
- A61B 6/5235: {combining images from the same or different ionising radiation imaging techniques, e.g. PET and CT}
- A61B 34/20: Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
- G06T 7/75: {involving models}
- A61B 6/032: {Transmission computed tomography [CT]}
- A61B 6/527: {using data from a motion artifact sensor}
- A61B 2034/2051: {Electromagnetic tracking systems}
- A61B 2034/2055: {Optical tracking systems}
- A61B 2034/2065: {Tracking using image or pattern recognition}
- G06T 2207/10072: Tomographic images
- G06T 2207/30021: Catheter; Guide wire
- G06T 2207/30096: Tumor; Lesion
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

The present teaching relates to surgical procedure assistance. In one example, a first image of an organ having a lesion is obtained prior to a surgical procedure. Information related to a pose of a surgical instrument at a first location with respect to the lesion is received from a sensor coupled with the surgical instrument. A visual environment having the lesion and the surgical instrument rendered therein is generated based on the first image and the information received from the sensor. A second image of the organ is obtained when the surgical instrument is moved to a second location with respect to the lesion during the surgical procedure. The second image captures the lesion and the surgical instrument. The pose of the surgical instrument and the lesion rendered in the visual environment are adjusted based, at least in part, on the second image.

## Claims

### Claim 1

A method, implemented on a computing device having at least one processor, storage, and a communication platform capable of connecting to a network for surgical procedure assistance, comprising:
obtaining a first image of an organ having a lesion prior to a surgical procedure;
receiving information, related to a pose of a surgical instrument at a first location with respect to the lesion, from a sensor coupled with the surgical instrument;
generating a visual environment having the lesion and the surgical instrument rendered therein based on the first image and the information received from the sensor;
obtaining a second image of the organ when the surgical instrument is disposed at a second location with respect to the lesion during the surgical procedure, wherein the second image captures the lesion and the surgical instrument;
mapping the lesion in the first image to the second image;
mapping the surgical instrument rendered in the visual environment to the second image; and
adjusting the pose of the surgical instrument and the lesion rendered in the visual environment, wherein the adjusting comprises compensating a difference in pose of the surgical instrument in the visual environment and in the second image, and compensating a position difference of the lesion in the first and second images.

### Claim 2 (depends on 1)

The method of claim 1, wherein the pose of the surgical instrument rendered in the visual environment is adjusted by compensating a position difference and/or an orientation difference of the surgical instrument in the visual environment and in the second image.

### Claim 3 (depends on 2)

The method of claim 2, wherein compensating the position difference and/or the orientation difference of the surgical instrument comprises:
computing a distance between a tip of the mapped surgical instrument and a tip of the surgical instrument in the second image; and
computing an angle between a direction of the mapped surgical instrument and a direction of the surgical instrument in the second image.

### Claim 4 (depends on 1)

The method of claim 1, wherein the pose of the surgical instrument rendered in the visual environment is adjusted based on a distance measure and an angle measure between the lesion and the surgical instrument in the second image.

### Claim 5 (depends on 4)

The method of claim 4, wherein the distance measure and the angle measure are computed based on the center of the lesion in the second image, a point in the second image at which the at least one surgical instrument enters a patient's body, and a tip of the at least one surgical instrument in the second image.

### Claim 6 (depends on 1)

The method of claim 1, wherein compensating the position difference of the lesion comprises:
computing a distance between the center of the mapped lesion from the first image and the center of the lesion in the second image.

### Claim 7 (depends on 1)

The method of claim 1, wherein the surgical instrument includes a needle.

### Claim 8 (depends on 1)

The method of claim 1, wherein the steps of receiving, generating, obtaining, and adjusting are repeated until the surgical instrument reaches the lesion during the surgical procedure.

### Claim 9

A system for surgical procedure assistance, comprising:
a tracking unit implemented by a processor and configured to receive information related to a pose of a surgical instrument at a first location with respect to a lesion from a sensor coupled with the surgical instrument;
a positioning unit implemented by a processor and configured to:
obtain a first image of an organ having the lesion prior to a surgical procedure,
generate a visual environment having the lesion and the surgical instrument rendered therein based on the first image and the information received from the sensor, and
obtain a second image of the organ when the surgical instrument is disposed at a second location with respect to the lesion during the surgical procedure, wherein the second image captures the lesion and the surgical instrument; and
a compensation unit implemented by the processor and configured to:
map the lesion in the first image to the second image,
map the surgical instrument rendered in the visual environment to the second image, and
adjust the pose of the surgical instrument and the lesion rendered in the visual environment by compensating a difference in pose of the surgical instrument in the visual environment and in the second image, and compensating a position difference of the lesion in the first and second images.

### Claim 10 (depends on 9)

The system of claim 9, wherein the one or more compensation units comprise a second compensation unit configured to compensate a position difference and/or an orientation difference of the surgical instrument in the visual environment and in the second image.

### Claim 11 (depends on 10)

The system of claim 10, wherein the second compensation unit is further configured to:
compute a distance between a tip of the mapped surgical instrument and a tip of the surgical instrument in the second image; and
compute an angle between a direction of the mapped surgical instrument and a direction of the surgical instrument in the second image.

### Claim 12 (depends on 9)

The system of claim 9, wherein the pose of the surgical instrument rendered in the visual environment is adjusted based on a distance measure and an angle measure between the lesion and the surgical instrument in the second image.

### Claim 13 (depends on 12)

The system of claim 12, wherein the distance measure and the angle measure are computed based on the center of the lesion in the second image, a point in the second image at which the at least one surgical instrument enters a patient's body, and a tip of the at least one surgical instrument in the second image.

### Claim 14 (depends on 9)

The system of claim 9, wherein the first compensation unit is further configured to:
compute a distance between the center of the mapped lesion from the first image and the center of the lesion in the second image.

### Claim 15 (depends on 9)

The system of claim 9, wherein the surgical instrument includes a needle.

### Claim 16

A non-transitory machine readable medium having information recorded thereon for surgical procedure assistance, wherein the information, when read by a machine, causes the machine to perform the steps of:
obtaining a first image of an organ having a lesion prior to a surgical procedure;
receiving information, related to a pose of a surgical instrument at a first location with respect to the lesion, from a sensor coupled with the surgical instrument;
generating a visual environment having the lesion and the surgical instrument rendered therein based on the first image and the information received from the sensor;
obtaining a second image of the organ when the surgical instrument is disposed at a second location with respect to the lesion during the surgical procedure, wherein the second image captures the lesion and the surgical instrument;
mapping the lesion in the first image to the second image;
mapping the surgical instrument rendered in the visual environment to the second image; and
adjusting the pose of the surgical instrument and the lesion rendered in the visual environment, wherein the adjusting comprises compensating a difference in pose of the surgical instrument in the visual environment and in the second image, and compensating a position difference of the lesion in the first and second images.

### Claim 17 (depends on 16)

The medium of claim 16, wherein the pose of the surgical instrument rendered in the visual environment is adjusted by compensating a position difference and/or an orientation difference of the surgical instrument in the visual environment and in the second image.

### Claim 18 (depends on 16)

The medium of claim 16, wherein the pose of the surgical instrument rendered in the visual environment is adjusted based on a distance measure and an angle measure between the lesion and the surgical instrument in the second image.

### Claim 19 (depends on 16)

The medium of claim 16, wherein the steps of receiving, generating, obtaining, and adjusting are repeated until the surgical instrument reaches the lesion during the surgical procedure.

### Claim 20

A method, implemented on a computing device having at least one processor, storage, and a communication platform capable of connecting to a network for instrument tracking and guidance, comprising:
obtaining a first image of an object prior to an operation;
receiving information, related to a pose of an instrument at a first location with respect to the object, from a sensor coupled with the instrument;
generating a visual environment having the object and the instrument rendered therein based on the first image and the information received from the sensor;
obtaining a second image capturing the object and the instrument when the instrument is disposed at a second location with respect to the object during the operation; and
adjusting the pose of the instrument and the object rendered in the visual environment based on mapping of the object in the first image to the second image, and mapping of the instrument rendered in the visual environment to the second image, the adjusting being performed by compensating a difference in pose of the instrument in the visual environment and in the second image, and compensating a position difference of the lesion in the first and second images.

### Claim 21 (depends on 20)

The method of claim 20, wherein the object is an organ including a lesion, and the instrument is one of a needle, a catheter, a probe, and a knife.