Using Grouped Linear Prediction and Accelerated Reinforcement Learning for Online Content Caching

Proactive caching is an effective way to alleviate peak-hour traffic congestion by prefetching popular contents at the wireless network edge. To maximize the caching efficiency requires the knowledge of content popularity profile, which however is often unavailable in advance. In this paper, we first propose a new linear prediction model, named grouped linear model (GLM) to estimate the future content requests based on historical data. Unlike many existing works that assumed the static content popularity profile, our model can adapt to the temporal variation of the content popularity in practical systems due to the arrival of new contents and dynamics of user preference. Based on the predicted content requests, we then propose a reinforcement learning approach with model-free acceleration (RLMA) for online cache replacement by taking into account both the cache hits and replacement cost. This approach accelerates the learning process in non-stationary environment by generating imaginary samples for Q-value updates. Numerical results based on real-world traces show that the proposed prediction and learning based online caching policy outperform all considered existing schemes.Comment: 6 pages, 4 figures, ICC 2018 worksho

A Bayesian Poisson-Gaussian Process Model for Popularity Learning in Edge-Caching Networks

Edge-caching is recognized as an efficient technique for future cellular networks to improve network capacity and user-perceived quality of experience. To enhance the performance of caching systems, designing an accurate content request prediction algorithm plays an important role. In this paper, we develop a flexible model, a Poisson regressor based on a Gaussian process, for the content request distribution. The first important advantage of the proposed model is that it encourages the already existing or seen contents with similar features to be correlated in the feature space and therefore it acts as a regularizer for the estimation. Second, it allows to predict the popularities of newly-added or unseen contents whose statistical data is not available in advance. In order to learn the model parameters, which yield the Poisson arrival rates or alternatively the content \textit{popularities}, we invoke the Bayesian approach which is robust against over-fitting. However, the resulting posterior distribution is analytically intractable to compute. To tackle this, we apply a Markov Chain Monte Carlo (MCMC) method to approximate this distribution which is also asymptotically exact. Nevertheless, the MCMC is computationally demanding especially when the number of contents is large. Thus, we employ the Variational Bayes (VB) method as an alternative low complexity solution. More specifically, the VB method addresses the approximation of the posterior distribution through an optimization problem. Subsequently, we present a fast block-coordinate descent algorithm to solve this optimization problem. Finally, extensive simulation results both on synthetic and real-world datasets are provided to show the accuracy of our prediction algorithm and the cache hit ratio (CHR) gain compared to existing methods from the literature

A Feature-Based Bayesian Method for Content Popularity Prediction in Edge-Caching Networks

Edge-caching is recognized as an efficient technique for future wireless cellular networks to improve network capacity and user-perceived quality of experience. Due to the random content requests and the limited cache memory, designing an efficient caching policy is a challenge. To enhance the performance of caching systems, an accurate content request prediction algorithm is essential. Here, we introduce a flexible model, a Poisson regressor based on a Gaussian process, for the content request distribution in stationary environments. Our proposed model can incorporate the content features as side information for prediction enhancement. In order to learn the model parameters, which yield the Poisson rates or alternatively content popularities, we invoke the Bayesian approach which is very robust against over-fitting. However, the posterior distribution in the Bayes formula is analytically intractable to compute. To tackle this issue, we apply a Monte Carlo Markov Chain (MCMC) method to approximate the posterior distribution. Two types of predictive distributions are formulated for the requests of existing contents and for the requests of a newly-added content. Finally, simulation results are provided to confirm the accuracy of the developed content popularity learning approach.Comment: arXiv admin note: substantial text overlap with arXiv:1903.0306

Online Learning Models for Content Popularity Prediction In Wireless Edge Caching

Caching popular contents in advance is an important technique to achieve the low latency requirement and to reduce the backhaul costs in future wireless communications. Considering a network with base stations distributed as a Poisson point process (PPP), optimal content placement caching probabilities are derived for known popularity profile, which is unknown in practice. In this paper, online prediction (OP) and online learning (OL) methods are presented based on popularity prediction model (PPM) and Grassmannian prediction model (GPM), to predict the content profile for future time slots for time-varying popularities. In OP, the problem of finding the coefficients is modeled as a constrained non-negative least squares (NNLS) problem which is solved with a modified NNLS algorithm. In addition, these two models are compared with log-request prediction model (RPM), information prediction model (IPM) and average success probability (ASP) based model. Next, in OL methods for the time-varying case, the cumulative mean squared error (MSE) is minimized and the MSE regret is analyzed for each of the models. Moreover, for quasi-time varying case where the popularity changes block-wise, KWIK (know what it knows) learning method is modified for these models to improve the prediction MSE and ASP performance. Simulation results show that for OP, PPM and GPM provides the best ASP among these models, concluding that minimum mean squared error based models do not necessarily result in optimal ASP. OL based models yield approximately similar ASP and MSE, while for quasi-time varying case, KWIK methods provide better performance, which has been verified with MovieLens dataset.Comment: 9 figure, 29 page