Method for optimizing the energy efficiency of wireless sensor network based on the assistance of unmanned aerial vehicle

A method for optimizing the energy efficiency of wireless sensor network (WSN) based on the assistance of unmanned aerial vehicle (UAV), comprising:

• training an agent which is used to determine a hover node for a UAV in simulation environment creating a WSN based on an actual deployment in simulation environment, where the WSN has A battery-supplied sensor nodes and a sink node, the sink node is a UAV;
• for sensor node ni, i=1, . . . , A, taking the other sensor nodes within its communication range as its neighbor nodes to create a neighbor node list Ninbr=[mi1, . . . , mi|nbri|] where mic is the cth neighbor node of sensor node ni, c=1, . . . , |nbri|, |nbri| is the number of neighbor nodes of sensor node ni;
• deploying an agent on the UAV to determine a hover node for the UAV, where the hover node is the sensor node above which the UAV hovers to collect the whole data of the WSN;
• training the agent by using an actor-critic reinforcement learning algorithm:
• 1.1). choosing any of the sensor nodes as the hover node, then based on the locations where the sensors deployed and the neighborhood relationships between the sensors, taking the distances between the sensors as weights to calculate a minimum spanning tree by using Kruskal algorithm, and then in the minimum spanning tree, taking the hover node as a root node to calculate each node's routing by using breadth-first-search algorithm;
• 1.2). for the different data that the sensor nodes need to collect, designing their probability distributions respectively based on existing prior knowledge to simulate the amount of data collected by sensor nodes in a real environment, and sending the collected data to the hover node according to their routings at intervals of α seconds, then sending the collected data to the UAV by the hover node, when the UAV hovers above the hover node, meanwhile, simulating the energy consumptions of sensor nodes;
• 1.3). determining a next hover node and generating a new routing scheme by the UAV when every β rounds of transmissions of the sensor nodes are completed, wherein the process of determining and generating are as follows:
• 1.3.1). determining a next hover node by the UAV
• 1.3.1.1). for sensor node ni, i=1, . . . , A, sending its residual energy to the UAV through current routing, and normalizing the residual energy in the UAV to obtain its normalized residual energy Wi, thus a residual energy vector {right arrow over (W)}=[W1, . . . , WA] of the sensor nodes is obtained;
• 1.3.1.2). obtaining a location vector {right arrow over (L)}=[(l11, l12), . . . , (lA1, lA2)] of the sensor nodes by the UAV according to the locations of the sensor nodes, where li1 and li2 correspond to the normalized horizontal coordinate and the normalized vertical coordinate of sensor node ni in a fixed coordinate system respectively;
• 1.3.1.3). concatenating residual energy vector {right arrow over (W)} and location vector {right arrow over (L)} to obtain a state vector {right arrow over (S)}={right arrow over (L)}+{right arrow over (W)} and sending the state vector {right arrow over (S)} to the decision network of the agent to calculate a probability vector {right arrow over (P)}=[p1, . . . , pA] by the UAV, where pi, i=1, . . . , A is the probability of choosing sensor node ni as a next hover node by the UAV;
• 1.3.1.4). randomly generating a floating number within the range of (0,1] by the UAV, wherein if the floating number fall in the jth interval of the cumulative distribution function vector of probability vector {right arrow over (P)}, the jth sensor node nj is chosen as the next hover node.
• 1.3.2). generating a new routing scheme by the UAV
• 1.3.2.1). for sensor node ni, i=1, . . . , A, using energy-balanced routing protocol (EBRP) algorithm to calculate its hybrid potential field list Ui=[ui1, . . . , ui|nbri|] according to its neighbor node list Ninbr by the UAV, where uic is the hybrid potential field between sensor node ni and its neighbor node mic, the value of uic stands for the preference of choosing neighbor node mic as parent node, the bigger the value is, the stronger the preference is;
• 1.3.2.2). for sensor node ni, i=1, . . . , A, calculating the distance to the next hover node according to its location by the UAV, sorting the sensor nodes in descending order by distance to obtain a node list {circumflex over (N)}=[{circumflex over (n)}1, . . . , {circumflex over (n)}A], where {circumflex over (n)}i is the ith sensor node in node list {circumflex over (N)};
• 1.3.2.3). maintaining an edge set E by the UAV, wherein the edges of edge set E is used to generate a spanning tree, the root node of the spanning tree is sensor node {circumflex over (n)}A=nj, initializing edge set E to an empty set;
• 1.3.2.4). traversing node list {circumflex over (N)} from sensor node {circumflex over (n)}1 to sensor node {circumflex over (n)}A to choose a parent node for each sensor node by the UAV, namely directing the sensor nodes to transmit data to the next hover node by choosing parent nodes for the sensor nodes from far to near distance to the next hover node:
• 1.3.2.4.1). letting i=1;
• 1.3.2.4.2). for sensor node {circumflex over (n)}i, if i=A, then performing step 1.3.2.5), if i≠A, then performing step 1.3.2.4.3);
• 1.3.2.4.3). wherein sensor node {circumflex over (n)}i corresponds to sensor node nk, sorting hybrid potential field list Ui of sensor node nk in descending order to obtain a list Ûk=[ûk1, . . . , ûk|nbrk|] where ûkc is the hybrid potential field between sensor node nk and its cth neighbor node {circumflex over (m)}kc after sorting;
• 1.3.2.4.4). traversing list Ûk from hybrid potential field ûk1 to hybrid potential field ûk|nbrk| to choose a neighbor node as the parent node of sensor node nk:
• 1.3.2.4.4.1). letting c=1;
• 1.3.2.4.4.2). for sensor node ûkc, checking whether a ring is formed after a corresponding edge ûk|nbrk| is added into edge set E, if yes, then performing step 1.3.2.4.4.3), otherwise, adding edge (nk, {circumflex over (m)}kc) into edge set E, then performing step 1.3.2.4.5);
• 1.3.2.4.4.3). if c=ûk|nbrk|, then calculating a minimum arborescence by using minimum directed spanning tree (MDST) algorithm and letting edge set E equal to a set of all edges in the minimum arborescence, then performing step 1.3.2.5), if c≠ûk|nbrk|, then letting c=c+1 and returning step 1.3.2.4.4.2);
• 1.3.2.4.5). letting i=i+1 and returning step 1.3.2.4.2);
• 1.3.2.5). generating a spanning tree according to edge set E, then in the spanning tree, taking sensor node n1, namely the next hover node as a root node to calculate each node's routing by using breadth-first-search algorithm;
• 1.3.3). sending each sensor node's routing in package form to its corresponding sensor node through current routing by the UAV, whereafter each sensor node sends data to the next hover node, namely sensor node nj through its received routing and the UAV flies to and hovers above the next hover node to collect data through the next hover node;
• 1.4). continuously performing step 1.3), until the energy of any sensor node is run out, the wireless sensor network is paralyzed, and then training the agent by using an actor-critic reinforcement learning algorithm, wherein the decision network of the agent is taken as an actor network, a critic network is set for instructing the learning of the actor network, state vector {right arrow over (S)} at the time of determining the next hover node is taken as the input of the actor network and the input of the critic network, the reward function in the process of training is calculated according to the lifetime of the wireless sensor network and the energy consumption of the whole sensor nodes, the calculating formula of the reward function is:
• R
• t
• =
• {
• R
• E
• ,
• the
• ⁢
• WSN
• ⁢
• is
• ⁢
• still
• ⁢
• running
• ⁢
• at
• ⁢
• the
• ⁢
• t
• t
• ⁢
• h
• ⁢
• next
• ⁢
• hover
• ⁢
• node
• ⁢
• determination
• R
• E
• +
• R
• T
• ,
• the
• ⁢
• WS
• ⁢
• N
• ⁢
• is
• ⁢
• paralyzed
• ⁢
• at
• ⁢
• the
• ⁢
• ⁢
• t
• t
• ⁢
• h
• ⁢
• next
• ⁢
• hover
• ⁢
• node
• ⁢
• determination
• where Rt is the value of the reward unction at tth next over node determination, RE is a value that is set according to the energy consumption of the whole sensor nodes between the tth next hover node determination and the (t−1)th next hover node determination, the higher the energy consumption of the whole sensor nodes is, the bigger the value of RE is, RT is a reward when the WSN is paralyzed and set according to the lifetime of the WSN, the longer the lifetime of the WSN is, the bigger the value of RT is the value;
• 1.5). repeating step 1.1) to step 1.4) to continuously update the weights of the actor network and the critic network until convergence;
• (2). deploying the UAV and the WSN into the real environment
• 2.1). randomly choosing a sensor node as the hover node and calculating each node's routing according to the method of step 1.1);
• 2.2). writing the location, neighbor nodes and routing of each sensor node into a configuration file of itself and a configuration file of the UAV respectively, deploying an agent used for determining a hover node into the UAV, the decision network of the agent is the trained decision network of the agent in simulation environment in step (1).
• 2.3). deploying the sensor nodes into the real environment according their locations, letting the UAV hover above the hover node;
• (3). continuously detecting the environment and collecting data, and sending the collected data to the hover node according to their routings at intervals of α seconds by all sensor sensors, then sending the collected data to the UAV by the hover node, when the UAV hovers above the hover node;
• (4). determining a next hover node by the UAV according to the method of step 1.3.1), when every β rounds of transmissions of the sensor nodes are completed, and generating a new routing scheme by the UAV according to the method of step 1.3.2), then sending each sensor node's routing to its corresponding sensor node through current routing by the UAV and letting the UAV flies to and hovers above the next hover node to collect data through the next hover node according to the method of step 1.3.3).