Mobile Edge Computation Offloading Using Game Theory and Reinforcement Learning

Due to the ever-increasing popularity of resource-hungry and delay-constrained mobile applications, the computation and storage capabilities of remote cloud has partially migrated towards the mobile edge, giving rise to the concept known as Mobile Edge Computing (MEC). While MEC servers enjoy the close proximity to the end-users to provide services at reduced latency and lower energy costs, they suffer from limitations in computational and radio resources, which calls for fair efficient resource management in the MEC servers. The problem is however challenging due to the ultra-high density, distributed nature, and intrinsic randomness of next generation wireless networks. In this article, we focus on the application of game theory and reinforcement learning for efficient distributed resource management in MEC, in particular, for computation offloading. We briefly review the cutting-edge research and discuss future challenges. Furthermore, we develop a game-theoretical model for energy-efficient distributed edge server activation and study several learning techniques. Numerical results are provided to illustrate the performance of these distributed learning techniques. Also, open research issues in the context of resource management in MEC servers are discussed

A Parallel Optimal Task Allocation Mechanism for Large-Scale Mobile Edge Computing

We consider the problem of intelligent and efficient task allocation mechanism in large-scale mobile edge computing (MEC), which can reduce delay and energy consumption in a parallel and distributed optimization. In this paper, we study the joint optimization model to consider cooperative task management mechanism among mobile terminals (MT), macro cell base station (MBS), and multiple small cell base station (SBS) for large-scale MEC applications. We propose a parallel multi-block Alternating Direction Method of Multipliers (ADMM) based method to model both requirements of low delay and low energy consumption in the MEC system which formulates the task allocation under those requirements as a nonlinear 0-1 integer programming problem. To solve the optimization problem, we develop an efficient combination of conjugate gradient, Newton and linear search techniques based algorithm with Logarithmic Smoothing (for global variables updating) and the Cyclic Block coordinate Gradient Projection (CBGP, for local variables updating) methods, which can guarantee convergence and reduce computational complexity with a good scalability. Numerical results demonstrate the effectiveness of the proposed mechanism and it can effectively reduce delay and energy consumption for a large-scale MEC system.Comment: 15 pages,4 figures, resource management for large-scale MEC. arXiv admin note: text overlap with arXiv:2003.1284

Joint Offloading and Resource Allocation in Vehicular Edge Computing and Networks

The emergence of computation intensive on-vehicle applications poses a significant challenge to provide the required computation capacity and maintain high performance. Vehicular Edge Computing (VEC) is a new computing paradigm with a high potential to improve vehicular services by offloading computation-intensive tasks to the VEC servers. Nevertheless, as the computation resource of each VEC server is limited, offloading may not be efficient if all vehicles select the same VEC server to offload their tasks. To address this problem, in this paper, we propose offloading with resource allocation. We incorporate the communication and computation to derive the task processing delay. We formulate the problem as a system utility maximization problem, and then develop a low-complexity algorithm to jointly optimize offloading decision and resource allocation. Numerical results demonstrate the superior performance of our Joint Optimization of Selection and Computation (JOSC) algorithm compared to state of the art solutions

Applications of Deep Reinforcement Learning in Communications and Networking: A Survey

This paper presents a comprehensive literature review on applications of deep reinforcement learning in communications and networking. Modern networks, e.g., Internet of Things (IoT) and Unmanned Aerial Vehicle (UAV) networks, become more decentralized and autonomous. In such networks, network entities need to make decisions locally to maximize the network performance under uncertainty of network environment. Reinforcement learning has been efficiently used to enable the network entities to obtain the optimal policy including, e.g., decisions or actions, given their states when the state and action spaces are small. However, in complex and large-scale networks, the state and action spaces are usually large, and the reinforcement learning may not be able to find the optimal policy in reasonable time. Therefore, deep reinforcement learning, a combination of reinforcement learning with deep learning, has been developed to overcome the shortcomings. In this survey, we first give a tutorial of deep reinforcement learning from fundamental concepts to advanced models. Then, we review deep reinforcement learning approaches proposed to address emerging issues in communications and networking. The issues include dynamic network access, data rate control, wireless caching, data offloading, network security, and connectivity preservation which are all important to next generation networks such as 5G and beyond. Furthermore, we present applications of deep reinforcement learning for traffic routing, resource sharing, and data collection. Finally, we highlight important challenges, open issues, and future research directions of applying deep reinforcement learning.Comment: 37 pages, 13 figures, 6 tables, 174 reference paper

Computation Rate Maximization in UAV-Enabled Wireless Powered Mobile-Edge Computing Systems

Mobile edge computing (MEC) and wireless power transfer (WPT) are two promising techniques to enhance the computation capability and to prolong the operational time of low-power wireless devices that are ubiquitous in Internet of Things. However, the computation performance and the harvested energy are significantly impacted by the severe propagation loss. In order to address this issue, an unmanned aerial vehicle (UAV)-enabled MEC wireless powered system is studied in this paper. The computation rate maximization problems in a UAV-enabled MEC wireless powered system are investigated under both partial and binary computation offloading modes, subject to the energy harvesting causal constraint and the UAV's speed constraint. These problems are non-convex and challenging to solve. A two-stage algorithm and a three-stage alternative algorithm are respectively proposed for solving the formulated problems. The closed-form expressions for the optimal central processing unit frequencies, user offloading time, and user transmit power are derived. The optimal selection scheme on whether users choose to locally compute or offload computation tasks is proposed for the binary computation offloading mode. Simulation results show that our proposed resource allocation schemes outperforms other benchmark schemes. The results also demonstrate that the proposed schemes converge fast and have low computational complexity.Comment: This paper has been accepted by IEEE JSA

A Survey on Mobile Edge Networks: Convergence of Computing, Caching and Communications

As the explosive growth of smart devices and the advent of many new applications, traffic volume has been growing exponentially. The traditional centralized network architecture cannot accommodate such user demands due to heavy burden on the backhaul links and long latency. Therefore, new architectures which bring network functions and contents to the network edge are proposed, i.e., mobile edge computing and caching. Mobile edge networks provide cloud computing and caching capabilities at the edge of cellular networks. In this survey, we make an exhaustive review on the state-of-the-art research efforts on mobile edge networks. We first give an overview of mobile edge networks including definition, architecture and advantages. Next, a comprehensive survey of issues on computing, caching and communication techniques at the network edge is presented respectively. The applications and use cases of mobile edge networks are discussed. Subsequently, the key enablers of mobile edge networks such as cloud technology, SDN/NFV and smart devices are discussed. Finally, open research challenges and future directions are presented as well

Energy-Efficient Joint Offloading and Wireless Resource Allocation Strategy in Multi-MEC Server Systems

Mobile edge computing (MEC) is an emerging paradigm that mobile devices can offload the computation-intensive or latency-critical tasks to the nearby MEC servers, so as to save energy and extend battery life. Unlike the cloud server, MEC server is a small-scale data center deployed at a wireless access point, thus it is highly sensitive to both radio and computing resource. In this paper, we consider an Orthogonal Frequency-Division Multiplexing Access (OFDMA) based multi-user and multi-MEC-server system, where the task offloading strategies and wireless resources allocation are jointly investigated. Aiming at minimizing the total energy consumption, we propose the joint offloading and resource allocation strategy for latency-critical applications. Through the bi-level optimization approach, the original NP-hard problem is decoupled into the lower-level problem seeking for the allocation of power and subcarrier and the upper-level task offloading problem. Simulation results show that the proposed algorithm achieves excellent performance in energy saving and successful offloading probability (SOP) in comparison with conventional schemes.Comment: 6 pages, 5 figures, to appear in IEEE ICC 2018, May 20-2

Heterogeneous Services Provisioning in Small Cell Networks with Cache and Mobile Edge Computing

In the area of full duplex (FD)-enabled small cell networks, limited works have been done on consideration of cache and mobile edge communication (MEC). In this paper, a virtual FD-enabled small cell network with cache and MEC is investigated for two heterogeneous services, high-data-rate service and computation-sensitive service. In our proposed scheme, content caching and FD communication are closely combined to offer high-data-rate services without the cost of backhaul resource. Computing offloading is conducted to guarantee the delay requirement of users. Then we formulate a virtual resource allocation problem, in which user association, power control, caching and computing offloading policies and resource allocation are jointly considered. Since the original problem is a mixed combinatorial problem, necessary variables relaxation and reformulation are conducted to transfer the original problem to a convex problem. Furthermore, alternating direction method of multipliers (ADMM) algorithm is adopted to obtain the optimal solution. Finally, extensive simulations are conducted with different system configurations to verify the effectiveness of the proposed scheme

Intelligent networking with Mobile Edge Computing: Vision and Challenges for Dynamic Network Scheduling

Mobile edge computing (MEC) has been considered as a promising technique for internet of things (IoT). By deploying edge servers at the proximity of devices, it is expected to provide services and process data at a relatively low delay by intelligent networking. However, the vast edge servers may face great challenges in terms of cooperation and resource allocation. Furthermore, intelligent networking requires online implementation in distributed mode. In such kinds of systems, the network scheduling can not follow any previously known rule due to complicated application environment. Then statistical learning rises up as a promising technique for network scheduling, where edges dynamically learn environmental elements with cooperations. It is expected such learning based methods may relieve deficiency of model limitations, which enhance their practical use in dynamic network scheduling. In this paper, we investigate the vision and challenges of the intelligent IoT networking with mobile edge computing. From the systematic viewpoint, some major research opportunities are enumerated with respect to statistical learning

Information-Centric Wireless Networks with Mobile Edge Computing

In order to better accommodate the dramatically increasing demand for data caching and computing services, storage and computation capabilities should be endowed to some of the intermediate nodes within the network. In this paper, we design a novel virtualized heterogeneous networks framework aiming at enabling content caching and computing. With the virtualization of the whole system, the communication, computing and caching resources can be shared among all users associated with different virtual service providers. We formulate the virtual resource allocation strategy as a joint optimization problem, where the gains of not only virtualization but also caching and computing are taken into consideration in the proposed architecture. In addition, a distributed algorithm based on alternating direction method of multipliers is adopted to solve the formulated problem, in order to reduce the computational complexity and signaling overhead. Finally, extensive simulations are presented to show the effectiveness of the proposed scheme under different system parameters