Edge network computing system with deep reinforcement learning based task scheduling

A task offloading method for an edge server comprising at least one processor in an edge network computing system, comprising:

• receiving a plurality of computing tasks from a terminal device;
• using a deep Q-learning neural network (DQN) with experience replay to select a cloud server to offload a portion of the plurality of computing tasks;
• sending the portion of the plurality of computing tasks to the cloud server;
• and
• executing the portion of the plurality of computing tasks and forwarding results of the portion of the plurality of computing tasks received from the cloud server to the terminal device;
• wherein:
• a first time is a time for completing a remaining portion of the plurality of computing tasks performed by one of the plurality of edge servers;
• a second time is a time for offloading and completing the portion of the plurality of computing tasks offloaded to the selected cloud server, and includes an uplink access latency, a data transmission time for transmitting the portion of the plurality of computing tasks from one of the plurality of edge servers to the selected cloud server, a computing time for completing the portion of the plurality of computing tasks by the selected cloud server, and a downlink access latency;
• when the plurality of computing tasks are independent of each other, a total time for completing the plurality of computing tasks is the greater of the first time and the second time, and an optimal task partitioning is achieved when the first time is equal to the second time;
• when the plurality of computing tasks are required to be performed sequentially, the total time for completing the plurality of computing tasks includes the greater of the first time and a sum of the uplink access latency and the data transmission time, the computing time, and the downlink access latency, and the optimal task partitioning is achieved when the first time is equal to the sum of the uplink access latency and the data transmission time;
• each edge server includes an agent of multi-agent reinforcement learning in a Markov decision process (MDP), and the agent learns from an environment, and takes action to maximize a long-term accumulative reward;
• a state space includes the uplink access latency, the downlink access latency, and a data rate of a backhaul link between the edge server and the cloud server, observed at a current time slot;
• an action space includes the plurality of cloud servers as candidates for task offloading, and a task partitioning ratio having a number of discrete values;
• and
• a reward includes minus the total time for completing the plurality of computing tasks.

A non-transitory computer-readable storage medium storing a computer program for task offloading, when being executed by an edge server comprising at least one processor, the computer program performing:

• receiving a plurality of computing tasks from a terminal device;
• using a deep Q-learning neural network (DQN) with experience replay to select a cloud server to offload a portion of the plurality of computing tasks;
• sending the portion of the plurality of computing tasks to the cloud server;
• and
• executing the portion of the plurality of computing tasks and forwarding results of the portion of the plurality of computing tasks received from the cloud server to the terminal device;
• wherein:
• a first time is a time for completing a remaining portion of the plurality of computing tasks performed by the edge server;
• a second time is a time for offloading and completing the portion of the plurality of computing tasks offloaded to the cloud server, and includes an uplink access latency, a data transmission time for transmitting the portion of the plurality of computing tasks from the edge server to the cloud server, a computing time for completing the portion of the plurality of computing tasks by the cloud server, and a downlink access latency;
• when the plurality of computing tasks are independent of each other, a total time for completing the plurality of computing tasks is the greater of the first time and the second time, and an optimal task partitioning is achieved when the first time is equal to the second time;
• when the plurality of computing tasks are required to be performed sequentially, the total time for completing the plurality of computing tasks includes the greater of the first time and a sum of the uplink access latency and the data transmission time, the computing time, and the downlink access latency, and the optimal task partitioning is achieved when the first time is equal to the sum of the uplink access latency and the data transmission time;
• each edge server includes an agent of multi-agent reinforcement learning in a Markov decision process (MDP), and the agent learns from an environment, and takes action to maximize a long-term accumulative reward;
• a state space includes the uplink access latency, the downlink access latency, and a data rate of a backhaul link between the edge server and the cloud server, observed at a current time slot;
• an action space includes a plurality of cloud servers as candidates for task offloading, and a task partitioning ratio having a number of discrete values;
• and
• a reward includes minus the total time for completing the plurality of computing tasks.