# US 2024/0005532 A1 · Dynamic tracking methods for in-vivo three-dimensional key point and in-vivo three-dimensional curve

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0005532 A1
- Document ID: US20240005532A1
- Status: Published application
- Kind: A1
- Application number: 18/346,331
- Filed: 2023-07-03
- Published: 2024-01-04
- Earliest claimed: 2022-07-04
- Assignees: Hefei University of Technology
- Applicants: Hefei University of Technology
- Inventors: Xiaojian LI; Ling LI; Shanlin YANG; Youtong TANTAI; Jin FANG; Jieyu ZHENG; Shuai DING; Lin ZHANG; Yanwei QU; Hua TANG; Liangkuan BI
- Web page: https://patentgrep.com/patent/US20240005532A1

## CPC

- G06T 7/248: {involving reference images or patches}
- G06V 10/235: {based on user input or interaction}
- G06T 3/02: Affine transformations (for image registration G06T3/147; for image mosaicing G06T3/4038)
- G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments
- G06T 7/50: Depth or shape recovery
- G06T 7/60: Analysis of geometric attributes
- G06T 7/74: {involving reference images or patches}
- G06V 10/462: {Salient features, e.g. scale invariant feature transforms [SIFT]}
- G06V 10/74: Image or video pattern matching; Proximity measures in feature spaces
- G06T 2207/10024: Color image
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/10068: Endoscopic image
- G06T 2207/30004: Biomedical image processing
- G06V 2201/03: Recognition of patterns in medical or anatomical images
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Dynamic tracking methods for an in-vivo three-dimensional key point and an in-vivo three-dimensional curve can include mapping a first local region to a first local point cloud and mapping a second local region to a second local point cloud according to a mapping relation between an endoscopic image and the point clouds; determining a first three-dimensional key point of a first two-dimensional key point on the first local point cloud, and acquiring a second three-dimensional key point on the second local point cloud through a coordinate transformation; and mapping the second three-dimensional key point back to the second local region, so as to acquire a second two-dimensional key point from a next image; and acquiring two-dimensional coordinates of a tracked key point by minimizing a preset optimization function in combination with an initial two-dimensional key point, and finally acquiring corresponding three-dimensional coordinates.

## Claims

### Claim at position 1

1-13. (canceled)

### Claim 14

A minimally invasive key site navigation oriented dynamic tracking method for an in-vivo three-dimensional key point, comprising:
S11, reading an endoscopic image, and acquiring a first two-dimensional key point from a current image according to selection of a doctor;
S12, tracking a first local region encompassing the first two-dimensional key point on the current image, acquiring a second local region from a next image, and determining an initial two-dimensional key point of the first two-dimensional key point on the next image;
S13, mapping the first local region to a first local point cloud and mapping the second local region to a second local point cloud according to a mapping relation between the endoscopic image and the point clouds, determining a first three-dimensional key point of the first two-dimensional key point on the first local point cloud, and acquiring a second three-dimensional key point on the second local point cloud through a coordinate transformation; and
S14, mapping the second three-dimensional key point back to the second local region, so as to acquire a second two-dimensional key point from the next image, acquiring two-dimensional coordinates of a tracked key point by minimizing a preset optimization function in combination with the initial two-dimensional key point, and finally acquiring corresponding three-dimensional coordinates.

### Claim 15 (depends on 14)

The tracking method for an in-vivo three-dimensional key point according to claim 14, wherein S12 comprises:
S121, defining a k−1th image as I(k−1)∈W×H×3, wherein W denotes a width of the endoscopic image, H denotes a height of the endoscopic image, I(k) denotes a kth image, and p(k−1) denotes a first two-dimensional key point, coordinates of which are (u1, v1); and
taking the first two-dimensional key point p(k−1) as a center pc(k−1) of the first local region, and determining a first local region R(k−1) according to a preset region shape and side length;
S122, performing feature matching on feature points of images I(k31 1), I(k) through an optical flow method, and acquiring a center pc(k), corresponding to pc(k−1), of the second local region from the image I(k),
p
c
(
k
)
=
p
c
(
k
-
1
)
+
1
m
⁢
∑
i
=
1
m
(
p
i
f
(
k
)
-
p
i
f
(
k
-
1
)
)
wherein pf(k)={p1f(k), p2f(k), . . . , pmf(k)} denotes a feature point on the image I(k), and m denotes the number of the feature point on the image I(k);
S123, determining a second local region (k) according to the center pc(k) and the preset region shape and side length; and
S124, directly determining an initial two-dimensional key point po(k) of the first two-dimensional key point p(k−1) on the next image still through the optical flow method.

### Claim 16 (depends on 15)

The dynamic tracking method for an in-vivo three-dimensional key point according to claim 15, wherein S13 comprises:
S131, estimating a depth of the endoscopic image to obtain a depth image corresponding to the endoscopic image, acquiring space information and color information of each pixel from the depth image and the endoscopic image respectively through reading in rows, and acquiring a first local point cloud (k−1) and a second local point cloud (k);
S132, determining a first three-dimensional key point P(k−1)∈3 of the first two-dimensional key point p(k−1) on the first local point cloud (k−1),
P(k−1)=ψ(p(k−1))
wherein ψ denotes a mapping relation from (k−1) to (k −1), which is recorded as (k−1)→(k −1);
S133, acquiring a feature point pair of the local regions (k−1), (k) through the optical flow method and recording same as X and Y, respectively, so that X and Y are in a coordinate transformation relation:
Y
=
[
A
t
]
[
X
1
]
wherein A∈3×3, t∈3×1, ω=[A t]T∈4×3 denote parameters of a fitting function, ω is acquirable from the following formula through least squares:
ω=([X1]T[X1])−1[X1]TY
and a transformation matrix of an affine transformation of P(k−1), P(k) is:
T
A
=
[
A
t
0
T
1
]
∈
ℝ
4
×
4
wherein 0T=(0,0,0); and
S134, performing a three-dimensional affine transformation on the first three-dimensional key point P(k−1), wherein a matrix form is:
P(k)=TAP(k−1)
and searching for a nearest point from the second local point cloud (k) to obtain an initial position of a second three-dimensional key point P(k).

### Claim 17 (depends on 14)

The dynamic tracking method for an in-vivo three-dimensional key point according to claim 14, wherein the optimization function in S14 is as follows:
=2−sift−optical
wherein denotes the optimization function;
sift denotes a cosine similarity of a scale invariant feature transform (SIFT) feature vector:
J
sift
(
Δ
⁢
u
,
Δ
⁢
v
,
k
)
=
ϕ
⁡
(
p
⁡
(
k
-
1
)
)
T
⁢
ϕ
⁡
(
p
u
+
Δ
⁢
u
,
v
+
Δ
⁢
v
(
k
)
)

ϕ
⁡
(
p
⁡
(
k
-
1
)
)

⁢

ϕ
⁡
(
p
u
+
Δ
⁢
u
,
v
+
Δ
⁢
v
(
k
)
)

wherein ϕ(p(k−1)) denotes a feature descriptor of the first two-dimensional key point p(k−1) and is a vector, ∥·∥ denotes a norm of the vector, (u, v) denote coordinates of the second two-dimensional key point p(k), pu+Δu,v+Δv(k) denotes a neighborhood point of the second two-dimensional key point p(k), Δu and Δv are coordinate offset of the second two-dimensional key point p(k), ϕ(pu+Δu,v+Δv(k)) denotes a feature descriptor of the neighborhood point pu+Δu,v+Δv(k) and is a vector, and
optical denotes an effect of optical flow information:
J
optical
(
Δ
⁢
u
,
Δ
⁢
v
,
k
)
=
a
T
⁢
b

a

⁢

b

wherein
a=(pu+Δu,v+Δv(k)−p(k))T,b=(po(k)−p(k))T.

### Claim 18 (depends on 15)

The dynamic tracking method for an in-vivo three-dimensional key point according to claim 15, wherein the optimization function in S14 is as follows:
=2−sift−optical
wherein denotes the optimization function;
sift denotes a cosine similarity of a scale invariant feature transform (SIFT) feature vector:
J
sift
(
Δ
⁢
u
,
Δ
⁢
v
,
k
)
=
ϕ
⁡
(
p
⁡
(
k
-
1
)
)
T
⁢
ϕ
⁡
(
p
u
+
Δ
⁢
u
,
v
+
Δ
⁢
v
(
k
)
)

ϕ
⁡
(
p
⁡
(
k
-
1
)
)

⁢

ϕ
⁡
(
p
u
+
Δ
⁢
u
,
v
+
Δ
⁢
v
(
k
)
)

wherein ϕ(p(k−1)) denotes a feature descriptor of the first two-dimensional key point p(k−1) and is a vector, ∥·∥ denotes a norm of the vector, (u, v) denote coordinates of the second two-dimensional key point p(k), pu+Δu,v+Δv(k) denotes a neighborhood point of the second two-dimensional key point p(k), Δu and Δv are coordinate offset of the second two-dimensional key point p(k), ϕ(pu+Δu,v+Δv(k)) denotes a feature descriptor of the neighborhood point pu+Δu,v+Δv(k) and is a vector, and
optical denotes an effect of optical flow information:
J
optical
(
Δ
⁢
u
,
Δ
⁢
v
,
k
)
=
a
T
⁢
b

a

⁢

b

wherein
a=(pu+Δu,v+Δv(k)—p(k))T,b=(po(k)−p(k))T.

### Claim 19 (depends on 16)

The dynamic tracking method for an in-vivo three-dimensional key point according to claim 16, wherein the optimization function in S14 is as follows:
=2−sift−optical
wherein denotes the optimization function;
sift denotes a cosine similarity of a scale invariant feature transform (SIFT) feature vector:
J
sift
(
Δ
⁢
u
,
Δ
⁢
v
,
k
)
=
ϕ
⁡
(
p
⁡
(
k
-
1
)
)
T
⁢
ϕ
⁡
(
p
u
+
Δ
⁢
u
,
v
+
Δ
⁢
v
(
k
)
)

ϕ
⁡
(
p
⁡
(
k
-
1
)
)

⁢

ϕ
⁡
(
p
u
+
Δ
⁢
u
,
v
+
Δ
⁢
v
(
k
)
)

wherein ϕ(p(k−1)) denotes a feature descriptor of the first two-dimensional key point p(k−1) and is a vector, ∥·∥ denotes a norm of the vector, (u, v) denote coordinates of the second two-dimensional key point p(k), pu+Δu,v+Δv(k) denotes a neighborhood point of the second two-dimensional key point p(k), Δu and Δv are coordinate offset of the second two-dimensional key point p(k), ϕ(pu+Δu,v+Δv(k)) denotes a feature descriptor of the neighborhood point pu+Δu,v+Δv(k) and is a vector, and
optical denotes an effect of optical flow information:
J
optical
(
Δ
⁢
u
,
Δ
⁢
v
,
k
)
=
a
T
⁢
b

a

⁢

b

wherein
a=(pu+Δu,v+Δv(k)−p(k))T,b=(po(k)−p(k))T.

### Claim 20 (depends on 17)

The dynamic tracking method for an in-vivo three-dimensional key point according to claim 17, wherein the acquiring two-dimensional coordinates of a tracked key point by minimizing a preset optimization function, and finally acquiring corresponding three-dimensional coordinates in S14 comprises:
acquiring Δû and Δ{circumflex over (v)} by traversing and searching for Δu and Δv, so as to satisfy the following expression:
(
Δ
⁢
u
ˆ
(
k
)
,
Δ
⁢
v
ˆ
(
k
)
)
=
argmin
(
Δ
⁢
u
,
Δ
⁢
v
)
⁢
(
J
⁡
(
Δ
⁢
u
,
Δv
,
k
)
)
and acquiring two-dimensional coordinates of a tracked key point pu+Δu,v+Δv(k) after ideal offset (Δû(k), Δ{circumflex over (v)}(k)) is obtained, and then finally acquiring corresponding three-dimensional coordinates according to the mapping relation between the endoscopic image and the point clouds.

### Claim 21 (depends on 14)

An electronic apparatus, comprising:
one or more processors;
a memory; and
one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the program comprises executing the dynamic tracking method for an in-vivo three-dimensional key point according to claim 14.

### Claim 22 (depends on 21)

The electronic apparatus according to claim 21, wherein S12 comprises:
S121, defining a k−1th image as I(k−1)∈W×H×3 wherein W denotes a width of the endoscopic image, H denotes a height of the endoscopic image, I(k) denotes a kth image, and p(k−1) denotes a first two-dimensional key point, coordinates of which are (u1, v1); and
taking the first two-dimensional key point p(k−1) as a center pc(k−1) of the first local region, and determining a first local region (k−1) according to a preset region shape and side length;
S122, performing feature matching on feature points of images I(k−1), I(k) through an optical flow method, and acquiring a center pc(k), corresponding to pc(k−1), of the second local region from the image I(k),
p
c
(
k
)
=
p
c
(
k
-
1
)
+
1
m
⁢
∑
i
=
1
m
(
p
i
f
(
k
)
-
p
i
f
(
k
-
1
)
)
wherein pf(k)={p1f(k), p2f(k), . . . , pmf(k)} denotes a feature point on the image I(k), and m denotes the number of the feature point on the image I(k);
S123, determining a second local region (k) according to the center pc(k) and the preset region shape and side length; and
S124, directly determining an initial two-dimensional key point po(k) of the first two-dimensional key point p(k−1) on the next image still through the optical flow method.

### Claim 23 (depends on 22)

The electronic apparatus according to claim 22, wherein S13 comprises:
S131, estimating a depth of the endoscopic image to obtain a depth image corresponding to the endoscopic image, acquiring space information and color information of each pixel from the depth image and the endoscopic image respectively through reading in rows, and acquiring a first local point cloud (k−1) and a second local point cloud (k);
S132, determining a first three-dimensional key point P(k−1)∈3 of the first two-dimensional key point p(k−1) on the first local point cloud (k−1),
P(k−1)=ψ(p(k−1))
wherein ψ denotes a mapping relation from (k−1) to (k−1), which is recorded as (k−1)→(k−1);
S133, acquiring a feature point pair of the local regions (k−1), (k) through the optical flow method and recording same as X and Y, respectively, so that X and Y are in a coordinate transformation relation:
Y
=
[
A
⁢
t
]
[
x
1
]
wherein A∈3×3, t∈3×1, ω=[A t]T∈4×3 denote parameters of a fitting function, ω is acquirable from the following formula through least squares:
ω=[X1]T[X1])−1[X1]TY
and a transformation matrix of an affine transformation of P(k−1), P(k) is:
T
A
=
[
A
t
0
T
1
]
∈
ℝ
4
×
4
wherein 0T=(0,0,0); and
S134, performing a three-dimensional affine transformation on the first three-dimensional key point P(k−1), wherein a matrix form is:
P(k)=TAP(k−1)
and searching for a nearest point from the second local point cloud (k) to obtain an initial position of a second three-dimensional key point P(k).

### Claim 24 (depends on 21)

The electronic apparatus according to claim 21, wherein the optimization function in S14 is as follows:
=2−sift−optical
wherein denotes the optimization function;
sift denotes a cosine similarity of a scale invariant feature transform (SIFT) feature vector:
J
sift
(
Δ
⁢
u
,
Δ
⁢
v
,
k
)
=
ϕ
⁡
(
p
⁡
(
k
-
1
)
)
T
⁢
ϕ
⁡
(
p
u
+
Δ
⁢
u
,
v
+
Δ
⁢
v
(
k
)
)

ϕ
⁡
(
p
⁡
(
k
-
1
)
)

|

ϕ
⁡
(
p
u
+
Δ
⁢
u
,
v
+
Δ
⁢
v
(
k
)
)

wherein ϕ(p(k−1)) denotes a feature descriptor of the first two-dimensional key point p(k−1) and is a vector, ∥·∥ denotes a norm of the vector, (u, v) denote coordinates of the second two-dimensional key point p(k), pu+Δu,v+Δv(k) denotes a neighborhood point of the second two-dimensional key point p(k), Δu and Δv are coordinate offset of the second two-dimensional key point p(k), ϕ(pu+Δu,v+Δv(k)) denotes a feature descriptor of the neighborhood point pu+Δu,v+Δv(k) and is a vector, and
optical denotes an effect of optical flow information:
J
optical
(
Δ
⁢
u
,
Δ
⁢
v
,
k
)
=
a
T
⁢
b

a
|

⁢

b

wherein
a=(pu+Δu,v+Δv(k)−p(k))T,b=(po(k)−p(k))T.

### Claim 25 (depends on 24)

The electronic apparatus according to claim 24, wherein the acquiring two-dimensional coordinates of a tracked key point by minimizing a preset optimization function, and finally acquiring corresponding three-dimensional coordinates in S14 comprises:
acquiring Δû and Δ{circumflex over (v)} by traversing and searching for Δu and Δv, so as to satisfy the following expression:
(
Δ
⁢
û
⁢
Δ
⁢
v
ˆ
(
k
)
)
=
arg
⁢
min
(
Δ
⁢
u
,
Δ
⁢
v
)
⁢
(
J
⁢
(
Δ
⁢
u
,
Δ
⁢
v
,
k
)
)
and acquiring two-dimensional coordinates of a tracked key point pu+Δu,v+Δv(k) after ideal offset (Δû(k), Δ{circumflex over (v)}(k)) is obtained, and then finally acquiring corresponding three-dimensional coordinates according to the mapping relation between the endoscopic image and the point clouds.

### Claim 26

A minimally invasive key trajectory navigation oriented dynamic tracking method for an in-vivo three-dimensional curve, comprising:
S21, reading an endoscopic image, acquiring an operation path curve from a current image according to selection of a doctor, and acquiring a plurality of first two-dimensional key points through which the operation path curve passes;
S22, tracking a first local region encompassing the first two-dimensional key point on the current image, and acquiring a second local region from a next image;
S23, mapping the first local region to a first local point cloud and mapping the second local region to a second local point cloud according to a mapping relation between the endoscopic image and the point clouds, determining a first three-dimensional key point of the first two-dimensional key point on the first local point cloud, and acquiring a second three-dimensional key point on the second local point cloud through a coordinate transformation;
S24, reducing a dimension of the first local point cloud to obtain a first two-dimensional point cloud, and acquiring a second two-dimensional key point of the first three-dimensional key point on the first two-dimensional point cloud;
reducing a dimension of the second local point cloud to obtain a second two-dimensional point cloud, and acquiring a third two-dimensional key point of the second three-dimensional key point on the second two-dimensional point cloud; and
acquiring two-dimensional coordinates of a tracked key point on the two-dimensional point cloud by minimizing a preset optimization function according to the second two-dimensional key point and the third two-dimensional key point; and
S25, acquiring three-dimensional coordinates of each tracked key point according to a mapping relation between the point clouds before and after dimension reduction, performing curve fitting, and finally obtaining a three-dimensional curve by means of tracking.

### Claim 27 (depends on 26)

The dynamic tracking method for an in-vivo three-dimensional curve according to claim 26, wherein S22 comprises:
S221, defining a k−1th image a I/(k−1)∈W×H×3, wherein W denotes a width of the endoscopic image, H denotes a height of the endoscopic image, I(k) denotes a kth image, and (k−1)={p1(k−1), . . . , pn(k−1)} denote a plurality of first two-dimensional key points; and
determining maximums and minimums of all the first two-dimensional key points on a u axis and a v axis of an image coordinate system, respectively, selecting a position of a median on each axis as pc(k−1)=(u1, v1)∈R2, and determining a first local region (k−1) according to a preset region shape and side length;
S222, performing feature matching on feature points of images I(k−1), I(k) through an optical flow method, and acquiring a center pc(k), corresponding to pc(k−1), of the second local region from the image I(k),
p
c
(
k
)
=
p
c
(
k
-
1
)
+
1
m
⁢
∑
i
=
1
m
(
p
i
f
(
k
)
-
p
i
f
(
k
-
1
)
)
where pf(k)={p1f(k), p2f(k), . . . , pmf(k)} denotes a feature point on the image I(k), and m denotes the number of the feature point on the image I(k); and
S223, determining a second local region (k) according to the center pc(k) and the preset region shape and side length.

### Claim 28 (depends on 27)

The dynamic tracking method for an in-vivo three-dimensional curve according to claim 27, wherein S23 comprises:
S231, estimating a depth of the endoscopic image to obtain a depth image corresponding to the endoscopic image, acquiring space information and color information of each pixel from the depth image and the endoscopic image respectively through reading in rows, and acquiring a first local point cloud (k−1) and a second local point cloud (k);
S232, determining a first three-dimensional key point (k−1)={P1(k−1), . . . , Pn(k−1)}⊂(k−1), Pi(k−1)∈3 of the first two-dimensional key point (k−1) on the first local point cloud (k−1),
Pi(k−1)=ψ(pi(k−1))
wherein ψ denotes a mapping relation from (k−1) to (k−1), which is recorded as (k−1)→(k−1);
S233, acquiring a feature point pair of the local regions (k−1), (k) through the optical flow method and recording same as X and Y, respectively, so that X and Y are in a coordinate transformation relation:
Y
=
[
A
⁢
t
]
[
X
1
]
[
x1
]
wherein A∈3×3, t∈3×1, ω=[A t]T∈4×3 denote parameters of a fitting function, ω is acquirable from the following formula through least squares:
ω=([X1]T[X1])−1[X1]TY
and a transformation matrix of an affine transformation of P(k−1), P(k) is:
T
A
=
[
A
t
0
T
1
]
∈
ℝ
4
×
4
wherein 0T=(0,0,0); and
S234, performing a three-dimensional affine transformation on the first three-dimensional key point P(k−1), and searching for a nearest point from the second local point cloud (k) to obtain an initial position of a second three-dimensional key point P(k), wherein
P(k)=TAP(k−1).

### Claim 29 (depends on 26)

The dynamic tracking method for an in-vivo three-dimensional curve according to claim 26, wherein the optimization function in S24 is as follows:
=1−sift+shape
where denotes the optimization function;
sift denotes a cosine similarity of an SIFT feature vector:
J
sift
(
T
i
(
k
)
)
=
ϕ
⁡
(
T
i
(
k
-
1
)
)
T
⁢
ϕ
⁡
(
T
i
(
k
)
)

ϕ
⁡
(
T
i
(
k
-
1
)
)

⁢

ϕ
⁡
(
T
i
(
k
)
)

wherein ϕ(Ti(k−1)) denotes a feature descriptor of an ith second two-dimensional key point Ti(k−1) and is a vector, ϕ(Ti(k)) denotes a feature descriptor of a neighborhood point of an ith third two-dimensional key point Ti(k) and is a vector, and ∥·∥ denotes a norm of the vector; and
shape denotes a difference in cosine values of included angles between adjacent key points on different curves:
𝔍
s
⁢
h
⁢
a
⁢
p
⁢
e
(
T
i
(
k
)
)
=
1
n
⁢
∑
i
=
0
n
❘
g
⁡
(
T
i
(
k
-
1
)
)
-
g
⁡
(
T
i
(
k
)
)
❘
"\[RightBracketingBar]"
wherein g(Ti(k)) is the cosine value of the included angle, which is specifically calculated as follows:
g
⁡
(
T
i
(
k
)
)
=
a
T
⁢
b

a

⁢

b

wherein a=(Ti+1(k)−Ti(k))T,b=(Ti−1(k)−Ti(k))T.

### Claim 30 (depends on 27)

The dynamic tracking method for an in-vivo three-dimensional curve according to any one of claim 27, wherein the optimization function in S24 is as follows:
=1−sift+shape
wherein denotes the optimization function;
sift denotes a cosine similarity of an SIFT feature vector:
J
sift
(
T
i
(
k
)
)
=
ϕ
⁡
(
T
i
(
k
-
1
)
)
T
⁢
ϕ
⁡
(
T
i
(
k
)
)

ϕ
⁡
(
T
i
(
k
-
1
)
)

⁢

ϕ
⁡
(
T
i
(
k
)
)

wherein ϕ(Ti(k−1)) denotes a feature descriptor of an ith second two-dimensional key point Ti(k−1) and is a vector, ϕ(Ti(k)) denotes a feature descriptor of a neighborhood point of an ith third two-dimensional key point Ti(k) and is a vector, and ∥·∥ denotes a norm of the vector; and
shape denotes a difference in cosine values of included angles between adjacent key points on different curves:
𝔍
s
⁢
h
⁢
a
⁢
p
⁢
e
(
T
i
(
k
)
)
=
1
n
⁢
∑
i
=
0
n
|
g
⁡
(
T
i
(
k
-
1
)
)
-
g
⁡
(
T
i
(
k
)
)
|
wherein g(Ti(k)) is the cosine value of the included angle, which is specifically calculated as follows:
g
⁡
(
T
i
(
k
)
)
=
a
T
⁢
b

a

⁢

b

wherein a=(Ti+1(k)−Ti(k))T,b=(Ti−1(k)−Ti(k))T.

### Claim 31 (depends on 29)

The dynamic tracking method for an in-vivo three-dimensional curve according to claim 29, wherein the acquiring two-dimensional coordinates of a tracked key point on the two-dimensional point cloud by minimizing a preset optimization function according to the second two-dimensional key point and the third two-dimensional key point in S24 comprises:
minimizing by traversing and searching for neighborhood points of all third two-dimensional key points Ti(k), so as to satisfy:
{circumflex over (T)}i=argmin(1−sift(Ti(k))+shape(Ti(k)))
and acquiring an ideal key point Ti set by minimizing the optimization function.

### Claim 32 (depends on 29)

The dynamic tracking method for an in-vivo three-dimensional curve according to claim 29, wherein the performing curve fitting, and finally obtaining a three-dimensional curve by means of tracking in S25 comprises:
performing interpolation fitting on a line through an equation of a B-spline curve, wherein a general equation of the B-spline curve is P(t)=Σi=0n Pi Fi,k(t):
wherein Pi denotes a feature point of a control curve, Fi,k(t) denotes a kth order B-spline basis function, and the three-dimensional curve is tracked through curve interpolation fitting.

### Claim 33 (depends on 29)

The dynamic tracking method for an in-vivo three-dimensional curve according to claim 29, wherein a first two-dimensional key point acquisition process in S21 comprises:
defining p={p0p, . . . , pjp, . . . , pl−1p} as a pixel through which the operation path curve passes;
wherein
for a point on the curve, a curvature of a jth pixel on the curve is:
K
j
=
v
j
+
α
-
v
j
u
j
+
α
-
u
j
,
j
=
0
,
α
,
2
⁢
α
⁢
…
,
j
<
l
-
1
,
wherein pj+αp=[uj+α vj+α] denotes coordinates of a j+αth pixel on the curve, and a denotes an interval number of the pixels when the curvature of the pixel is solved; and
for curvatures of two consecutive pixels, when |Kj+α−Kj|≥ε, ε denotes a curvature threshold; and determining all first two-dimensional key points ={p1, . . . , pi, . . . , pn} in combination with pj+αp as a key point on the operation path curve and a start point and an end point of the curve, wherein n denotes a total number of the first two-dimensional key points.
