Self-powered integrated sensing and communication interactive method of high-speed railway based on hierarchical deep reinforcement learning

A self-powered integrated sensing and communication (ISAC) interactive method of a high-speed train based on hierarchical deep reinforcement learning (HDRL), comprising the following steps:

• constructing an integrated system framework for passive sensing and communication of the high-speed train, wherein the integrated system framework comprises high-speed train carriages and base stations (BSs) in remote communication with the high-speed train carriages, the high-speed train carriages each comprise an access point (AP) and passive sensor, the passive sensors are configured for harvesting wireless radio frequency (RF) energy from the AP and transmitting sensed train state information to the AP, and the AP is configured for receiving the train state information and communicating remotely with the BSs;
• establishing an information sensing and remote communication integrated model, wherein the information sensing and remote communication integrated model consists of a RF energy harvesting model for the passive sensors, an information sensing model for the train APs and remote communication model between the APs and the BSs, the information sensing and remote communication integrated model is configured for using a gaussian mixture model (GMM) clustering method to divide a remote communication handover triggering area and obtain reference handover triggering points, such that the high-speed train carriages can change the BSs in communication with the APs in the reference handover triggering points during moving of the high-speed train;
• and
• formulation of an integrated optimization problem:
• establishing a joint optimization model with task completion time minimization as an objective function, training the joint optimization model by applying an option-based HDRL algorithm, and solving an optimal configuration of the integrated system framework to acquire an optimal interaction policy, where the task completion time comprises time duration for the passive sensors harvesting energy, time duration for the the AP sensing information and time duration for the AP communicating with the corresponding remote BS, and the joint optimization model represents a dynamic autonomous switching process of energy transmission, information sensing and remote communication of the high-speed train carriages.