Joint Long-Term Cache Updating and Short-Term Content Delivery in Cloud-Based Small Cell Networks

Explosive growth of mobile data demand may impose a heavy traffic burden on fronthaul links of cloud-based small cell networks (C-SCNs), which deteriorates users' quality of service (QoS) and requires substantial power consumption. This paper proposes an efficient maximum distance separable (MDS) coded caching framework for a cache-enabled C-SCNs, aiming at reducing long-term power consumption while satisfying users' QoS requirements in short-term transmissions. To achieve this goal, the cache resource in small-cell base stations (SBSs) needs to be reasonably updated by taking into account users' content preferences, SBS collaboration, and characteristics of wireless links. Specifically, without assuming any prior knowledge of content popularity, we formulate a mixed timescale problem to jointly optimize cache updating, multicast beamformers in fronthaul and edge links, and SBS clustering. Nevertheless, this problem is anti-causal because an optimal cache updating policy depends on future content requests and channel state information. To handle it, by properly leveraging historical observations, we propose a two-stage updating scheme by using Frobenius-Norm penalty and inexact block coordinate descent method. Furthermore, we derive a learning-based design, which can obtain effective tradeoff between accuracy and computational complexity. Simulation results demonstrate the effectiveness of the proposed two-stage framework.Comment: Accepted by IEEE Trans. Commu

Multi-Agent Reinforcement Learning for Cooperative Coded Caching via Homotopy Optimization

Introducing cooperative coded caching into small cell networks is a promising approach to reducing traffic loads. By encoding content via maximum distance separable (MDS) codes, coded fragments can be collectively cached at small-cell base stations (SBSs) to enhance caching efficiency. However, content popularity is usually time-varying and unknown in practice. As a result, cache contents are anticipated to be intelligently updated by taking into account limited caching storage and interactive impacts among SBSs. In response to these challenges, we propose a multi-agent deep reinforcement learning (DRL) framework to intelligently update cache contents in dynamic environments. With the goal of minimizing long-term expected fronthaul traffic loads, we first model dynamic coded caching as a cooperative multi-agent Markov decision process. Owing to MDS coding, the resulting decision-making falls into a class of constrained reinforcement learning problems with continuous decision variables. To deal with this difficulty, we custom-build a novel DRL algorithm by embedding homotopy optimization into a deep deterministic policy gradient formalism. Next, to empower the caching framework with an effective trade-off between complexity and performance, we propose centralized, partially and fully decentralized caching controls by applying the derived DRL approach. Simulation results demonstrate the superior performance of the proposed multi-agent framework.Comment: Submitted to IEEE for possible publicatio