Methods for edge cache replacement based on deep reinforcement learning

A method for edge cache replacement based on deep reinforcement learning, comprising:

• S1, constructing a system network model;
• S2, obtaining a caching state of a mobile edge computing (MEC) server in the system network model, and calculating a total delay for a single user to obtain a cached content based on the caching state of the MEC server;
• S3, calculating a total delay for all users to obtain cached contents based on the total delay for the single user to obtain the cached content;
• S4, constructing an optimization objective function based on the total delay for all users to obtain the cached contents, the optimization objective function being as follows:
• min
• st
• f
• time
• all
• st
• .
• ψ
• ⁡
• (
• 1
• )
• :
• ∑
• f
• =
• 1
• F
• Z
• f
• ⁢
• st
• f
• ≤
• Cap
• M
• ⁢
• E
• ⁢
• C
• ,
• ∀
• f
• ∈
• File
• ψ
• ⁢
• (
• 2
• )
• :
• st
• f
• ∈
• {
• 0
• ,
• 1
• }
• ,
• ∀
• f
• ∈
• File
• wherein Q denotes a total count of all the users, denotes a count of content requests by the single user within a time interval τ;
• F denotes a total count of the cached contents obtained by all the users, Zf denotes a size of the cached content f obtained by the single user, stf denotes a caching state of the cached content f, CapMEC denotes a storage capacity of the MEC server, File denotes a set of all content files, a constraint condition ψ(1) means that a total volume of cached contents in the MEC server does not exceed the storage capacity;
• and a constraint condition ψ(2) is a constraint of the caching state of the cached content f;
• S5, formulating a Markov decision process (MDP) according to the optimization objective function;
• and
• S6, replacing a cached content in the MEC server using a dynamic content replacement algorithm based on deep reinforcement learning according to the Markov decision process.