Stochastic Game based Cooperative Alternating Q-Learning Caching in Dynamic D2D Networks

Edge caching has become an effective solution to cope with the challenges brought by the massive content delivery in cellular networks. In device-to-device (D2D) enabled caching cellular networks with time-varying content popularity distribution and user terminal (UT) location, we model these dynamic networks as a stochastic game to design a cooperative cache placement policy. The cache placement reward of each UT is defined as the caching incentive minus the transmission power cost for content caching and sharing. We consider the long-term cache placement reward of all UTs in this stochastic game. In an effort to solve the stochastic game problem, we propose a multi-agent cooperative alternating Q-learning (CAQL) based cache placement algorithm. The caching control unit is defined to execute the proposed CAQL, in which, the cache placement policy of each UT is alternatively updated according to the stable policy of other UTs during the learning process, until the stable cache placement policy of all the UTs in the cell is obtained. We discuss the convergence and complexity of CAQL, which obtains the stable cache placement policy with low space complexity. Simulation results show that the proposed algorithm can effectively reduce the backhaul load and the average content access delay in dynamic networks

Two Time-Scale Caching Placement and User Association in Dynamic Cellular Networks

With the rapid growth of data traffic in cellular networks, edge caching has become an emerging technology for traffic offloading. We investigate the caching placement and content delivery in cache-enabling cellular networks. To cope with the time-varying content popularity and user location in practical scenarios, we formulate a long-term joint dynamic optimization problem of caching placement and user association for minimizing the content delivery delay which considers both content transmission delay and content update delay. To solve this challenging problem, we decompose the optimization problem into two sub-problems, the user association sub-problem in a short time scale and the caching placement in a long time scale. Specifically, we propose a low complexity user association algorithm for a given caching placement in the short time scale. Then we develop a deep deterministic policy gradient based caching placement algorithm which involves the short time-scale user association decisions in the long time scale. Finally, we propose a joint user association and caching placement algorithm to obtain a sub-optimal solution for the proposed problem. We illustrate the convergence and performance of the proposed algorithm by simulation results. Simulation results show that compared with the benchmark algorithms, the proposed algorithm reduces the long-term content delivery delay in dynamic networks effectively

Mobile Edge Computing

This is an open access book. It offers comprehensive, self-contained knowledge on Mobile Edge Computing (MEC), which is a very promising technology for achieving intelligence in the next-generation wireless communications and computing networks. The book starts with the basic concepts, key techniques and network architectures of MEC. Then, we present the wide applications of MEC, including edge caching, 6G networks, Internet of Vehicles, and UAVs. In the last part, we present new opportunities when MEC meets blockchain, Artificial Intelligence, and distributed machine learning (e.g., federated learning). We also identify the emerging applications of MEC in pandemic, industrial Internet of Things and disaster management. The book allows an easy cross-reference owing to the broad coverage on both the principle and applications of MEC. The book is written for people interested in communications and computer networks at all levels. The primary audience includes senior undergraduates, postgraduates, educators, scientists, researchers, developers, engineers, innovators and research strategists

Holistic resource management in UAV-assisted wireless networks

Unmanned aerial vehicles (UAVs) are considered as a promising solution to assist terrestrial networks in future wireless networks (i.e., beyond fifth-generation (B5G) and sixth-generation (6G)). The convergence of various technologies requires future wireless networks to provide multiple functionalities, including communication, computing, control, and caching (4C), necessary for applications such as connected robotics and autonomous systems. The majority of existing works consider the developments in 4C individually, which limits the cooperation among 4C for potential gains. UAVs have been recently introduced to supplement mobile edge computing (MEC) in terrestrial networks to reduce network latency by providing mobile resources at the network edge in future wireless networks. However, compared to ground base stations (BSs), the limited resources at the network edge call for holistic management of the resources, which requires joint optimization. We provide a comprehensive review of holistic resource management in UAV-assisted wireless networks. Integrated resource management considers the challenges associated with aerial networks (such as three-dimensional (3D) placement of UAVs, trajectory planning, channel modelling, and backhaul connectivity) and terrestrial networks (such as limited bandwidth, power, and interference). We present architectures (source-UAV-destination and UAV-destination architecture) and 4C in UAV-assisted wireless networks. We then provide a detailed discussion on resource management by categorizing the optimization problems into individual or combinations of two (communication and computation) or three (communication, computation and control). Moreover, solution approaches and performance metrics are discussed and analyzed for different objectives and problem types. We formulate a mathematical framework for holistic resource management to minimize the linear combination of network latency and cost for user association while guaranteeing the offloading, computing, and caching constraints. Binary decision variables are used to allocate offloading and computing resources. Since the decision variables are binary and constraints are linear, the formulated problem is a binary linear programming problem. We propose a heuristic algorithm based on the interior point method by exploiting the optimization structure of the problem to get a sub-optimal solution with less complexity. Simulation results show the effectiveness of the proposed work when compared to the optimal results obtained using branch and bound. Finally, we discuss insight into the potential future research areas to address the challenges of holistic resource management in UAV-assisted wireless networks

A survey of multi-access edge computing in 5G and beyond : fundamentals, technology integration, and state-of-the-art

Driven by the emergence of new compute-intensive applications and the vision of the Internet of Things (IoT), it is foreseen that the emerging 5G network will face an unprecedented increase in traffic volume and computation demands. However, end users mostly have limited storage capacities and finite processing capabilities, thus how to run compute-intensive applications on resource-constrained users has recently become a natural concern. Mobile edge computing (MEC), a key technology in the emerging fifth generation (5G) network, can optimize mobile resources by hosting compute-intensive applications, process large data before sending to the cloud, provide the cloud-computing capabilities within the radio access network (RAN) in close proximity to mobile users, and offer context-aware services with the help of RAN information. Therefore, MEC enables a wide variety of applications, where the real-time response is strictly required, e.g., driverless vehicles, augmented reality, robotics, and immerse media. Indeed, the paradigm shift from 4G to 5G could become a reality with the advent of new technological concepts. The successful realization of MEC in the 5G network is still in its infancy and demands for constant efforts from both academic and industry communities. In this survey, we first provide a holistic overview of MEC technology and its potential use cases and applications. Then, we outline up-to-date researches on the integration of MEC with the new technologies that will be deployed in 5G and beyond. We also summarize testbeds and experimental evaluations, and open source activities, for edge computing. We further summarize lessons learned from state-of-the-art research works as well as discuss challenges and potential future directions for MEC research