1 research outputs found
When Deep Reinforcement Learning Meets Federated Learning: Intelligent Multi-Timescale Resource Management for Multi-access Edge Computing in 5G Ultra Dense Network
Ultra-dense edge computing (UDEC) has great potential, especially in the 5G
era, but it still faces challenges in its current solutions, such as the lack
of: i) efficient utilization of multiple 5G resources (e.g., computation,
communication, storage and service resources); ii) low overhead offloading
decision making and resource allocation strategies; and iii) privacy and
security protection schemes. Thus, we first propose an intelligent ultra-dense
edge computing (I-UDEC) framework, which integrates blockchain and Artificial
Intelligence (AI) into 5G ultra-dense edge computing networks. First, we show
the architecture of the framework. Then, in order to achieve real-time and low
overhead computation offloading decisions and resource allocation strategies,
we design a novel two-timescale deep reinforcement learning (\textit{2Ts-DRL})
approach, consisting of a fast-timescale and a slow-timescale learning process,
respectively. The primary objective is to minimize the total offloading delay
and network resource usage by jointly optimizing computation offloading,
resource allocation and service caching placement. We also leverage federated
learning (FL) to train the \textit{2Ts-DRL} model in a distributed manner,
aiming to protect the edge devices' data privacy. Simulation results
corroborate the effectiveness of both the \textit{2Ts-DRL} and FL in the I-UDEC
framework and prove that our proposed algorithm can reduce task execution time
up to 31.87%.Comment: Accepted by IEEE IoT