US 11,634,227 B2 · Landing tracking control method and system based on lightweight twin network and unmanned aerial vehicle

Abstract

A landing tracking control method comprises the following contents: a tracking model training stage and an unmanned aerial vehicle real-time tracking stage. The landing tracking control method extracts a network Snet by using a lightweight feature and makes modification, so that an extraction speed of the feature is increased to better meet a real-time requirement. Weight allocation on the importance of channel information is carried out to differentiate effective features more purposefully and utilize the features, so that the tracking precision is improved. In order to improve a training effect of the network, a loss function of an RPN network is optimized, a regression precision of a target frame is measured by using CIOU, and meanwhile, calculation of classified loss function is adjusted according to CIOU, and a relation between a regression network and classification network is enhanced.

A landing tracking control method based on a lightweight twin network, the method comprising the following contents:

in a tracking model training stage:
a1. Extracting a target image in a target template, and extracting a search image in a search area;
inputting the target image and the search image into two same lightweight SNet feature extraction modules, and extracting a search feature and a target feature by using a lightweight network SNet in the lightweight SNet feature extraction modules;
a2. Adjusting weights of the search feature and the target feature by a feature weight adjusting module to obtain an adjusted search feature and an adjusted target feature;
a3. Inputting the adjusted search feature and the adjusted target feature into an enhanced feature module to obtain an enhanced search feature and an enhanced target feature by a feature enhancing operation;
a4. Inputting the enhanced search feature and the enhanced target feature into a same RPN network to determine a type and a position of a target;
a5. Measuring a regression precision of a target frame by using CIOU, wherein when a CIOU value is relatively great indicating that the target frame is positioned precisely;
and as far as a sample with a high frame precision is concerned, a classification loss value and a frame loss value thereof are increased, and otherwise, the classification loss value and the frame loss value thereof are decreased;
and
a6. Carrying out multiple circuit training to finally obtain a tracking model;
and
in an unmanned aerial vehicle real-time tracking stage,
b1. Carrying out frame identifying operation on a camera carried by the unmanned aerial vehicle;
b2. Introducing frame identifying information into the tracking model and identifying a target;
and
b3. Judging whether the target is positioned successfully or not, if so, carrying out forecasting search on the target by means of Kalman algorithm and returning to the step b1 to operate;
and if not, adjusting and expanding the search range and returning to the step b1 to operate;
wherein in the step a1, after a deep separated convoluting operation of the lightweight network SNet, three search features and three target features are obtained;
the weight adjusting operation in the step a2 comprises the following contents:
compressing a feature pattern by the search feature and the target feature by utilizing global max-pooling first;
then training a set of stipulated parameters to represent weight of a channel feature pattern then via full convolutional and nonlinear activation operations, and finally, multiplying features of original channels by weight values obtained by the full convolutional and nonlinear activation operations to obtain an adjusted search feature pattern and an adjusted target feature pattern;
the steps a3 and a4 further comprise the following contents:
enhancing the extracted adjusted search feature and the adjusted target feature based on a feature pyramid, carrying out feature fusion, and inputting the fused feature into the RPN network to determine the type and the position of the target from three dimensions;
the step a5 further comprises the following contents:
For an optimization problem of the loss function, measuring a regression accuracy of the target frame by using CIOU, wherein when the CIOU value is relatively great, it is indicated that the target frame is positioned precisely;
and as far as a sample with a high frame precision is concerned, the classification loss value and the frame loss value thereof are increased, and otherwise, the classification loss value and the frame loss value thereof are decreased;
IOU in the CIOU is specifically as follows:
IOU
=
❘
"\[LeftBracketingBar]"
A
⋂
b
❘
"\[RightBracketingBar]"
❘
"\[LeftBracketingBar]"
A
⋃
b
❘
"\[RightBracketingBar]"
a
=
v
1
-
IOU
+
v
v
=
4
π
2
⁢
(
arc
⁢
tan
⁢
w
gt
h
gt
-
arc
⁢
tan
⁢
w
h
)
2
(
3
)
CIOU
=
IOU
-
ρ
2
(
b
,
b
gt
)
c
2
-
a
*
v
(
4
)
Loss
b
=
1
-
C
IOU
(
5
)
Loss
c
=
-
[
g
*
ln
⁢
p
+
(
1
-
g
)
*
ln
⁡
(
1
-
p
)
]
(
6
)
p
=
Pre
object
+
C
IOU
*
(
1
-
Pre
object
)
,
(
7
)
wherein A represents a true frame, B represents a prediction frame, w, h, b, wgt, hgtand bgt respectively represent widths, heights and centers of the true frame and the prediction frame, ρ2(b,bgt) represents an Euclidean distance between the center points of the true frame and prediction frame, c represents a diagonal length of the minimum rectangle containing the true frame and prediction frame true frame and prediction frame, Lossb represents a frame loss function, Lossc represents a classification loss function, g represents whether it is the target or not, if so, it is 1, and if not, it is 0;
Preobject represents a classification probability predicted by the RPN network, and when the regression precision of the prediction frame is high, it is considered that classification is relatively reliable, and the classification prediction probability thereof is increased;
the step b3 further comprises the following contents:
for a re-positioning problem after the target disappears, predicting a next frame position of the target by using Kalman algorithm, and locally detecting the target by taking a prediction result as a center;
if the target tracking in some continuous frames are missing, adding one of the length and width of the search area additionally by taking the prediction position result of Kalman filtering as a center, wherein the item is increased along with increment of time before the length and width of the search area exceed a video frame itself, and if the target is not detected for a long time, a final detection area is expanded to a whole image;
and
the step b3 further comprises the following contents:
w
search
=
w
pre
+
a
*
v
w
*
t
*
(
w
frame
-
w
pre
v
w
)
(
8
)
h
search
=
h
pre
+
a
*
v
h
*
t
*
(
h
frame
-
h
pre
v
h
)
(
9
)
wsearchand hsearch are width and height of the search area, wpre and hpre are width and height of a Kalman prediction area, wframe, andframe are width and height of the video frame, vw and vh are Kalman average moving speeds in transverse and longitudinal directions of the target in a previous n frame image, a is a constant that controls an area expanding speed, and t is a video frame number counted from start of tracking loss.

210 more elements

Landing tracking control method and system based on lightweight twin network and unmanned aerial vehicle

Life of this patent

Landing tracking control method and system based on lightweight twin network and unmanned aerial vehicle

Life of this patent

Abstract

Claims3