Security and Cost-Aware Computation Offloading via Deep Reinforcement Learning in Mobile Edge Computing

© 2019 Binbin Huang et al. With the explosive growth of mobile applications, mobile devices need to be equipped with abundant resources to process massive and complex mobile applications. However, mobile devices are usually resource-constrained due to their physical size. Fortunately, mobile edge computing, which enables mobile devices to offload computation tasks to edge servers with abundant computing resources, can significantly meet the ever-increasing computation demands from mobile applications. Nevertheless, offloading tasks to the edge servers are liable to suffer from external security threats (e.g., snooping and alteration). Aiming at this problem, we propose a security and cost-aware computation offloading (SCACO) strategy for mobile users in mobile edge computing environment, the goal of which is to minimize the overall cost (including mobile device's energy consumption, processing delay, and task loss probability) under the risk probability constraints. Specifically, we first formulate the computation offloading problem as a Markov decision process (MDP). Then, based on the popular deep reinforcement learning approach, deep Q-network (DQN), the optimal offloading policy for the proposed problem is derived. Finally, extensive experimental results demonstrate that SCACO can achieve the security and cost efficiency for the mobile user in the mobile edge computing environment

Mobile Edge Computing: From Task Load Balancing to Real-World Mobile Sensing Applications

University of Technology Sydney. Faculty of Engineering and Information Technology.With the rapid development of mobile computing technologies and the Internet of Things, there has been an increasing rise of capable and affordable edge devices that can provide in-proximity computing services for mobile users. Moreover, a massive amount of mobile edge computing (MEC) systems have been developed to enhance various aspects of people's daily life, including big mobile data, healthcare, intelligent transportation, connected vehicles, smart building control, indoor localization, and many others. Although MEC systems can provide mobile users with swift computing services and conserve devices' energy by processing their tasks, we confront significant research challenges in several perspectives, including resource management, task scheduling, service placement, application development, etc. For instance, computation offloading in MEC would significantly benefit mobile users and bring new challenges for service providers. Unbalance and inefficiency are the two challenging issues when making decisions on computation offloading among MEC servers. On the other hand, it is unprecedented to design and implement novel and practical applications for edge-assisted mobile computing and mobile sensing. The power of mobile edge computing has not been fully unleashed yet from theoretical and practical perspectives. In this thesis, to address the above challenges from both theoretical and practical perspectives, we present four research studies within the scope of MEC, including load balancing of computation task loading, fairness in workload scheduling, edge-assisted wireless sensing, and cross-domain learning for real-world edge sensing. The thesis consists of two major parts as follows. In the first part of this thesis, we investigate load balancing issues of computation offloading in MEC. First, we present a novel collaborative computation offloading mechanism for balanced mobile cloudlet networks. Then, a fairness-oriented task offloading scheme for IoT applications of MEC is further devised. The proposed computation offloading mechanisms incorporate algorithmic theories with the random mobility and opportunistic encounters of edge servers, thereby processing computation offloading for load balancing in a distributed manner. Through rigorous theoretical analyses and extensive simulations with real-world trace datasets, the proposed methods have demonstrated desirable results of significantly balanced computation offloading, showing great potential to be applied in practice. In the second part of this thesis, beyond theoretical perspectives, we further investigate two novel implementations with mobile edge computing, including edge-assisted wireless crowdsensing for outdoor RSS maps, and urban traffic prediction with cross-domain learning. We implement our ideas with the iMap system and the BuildSenSys system, and further demonstrate demos with real-world datasets to show the effectiveness of proposed applications. We believe that the above algorithms and applications hold great promise for future technological advancement in mobile edge computing

Task Graph offloading via Deep Reinforcement Learning in Mobile Edge Computing

Various mobile applications that comprise dependent tasks are gaining widespread popularity and are increasingly complex. These applications often have low-latency requirements, resulting in a significant surge in demand for computing resources. With the emergence of mobile edge computing (MEC), it becomes the most significant issue to offload the application tasks onto small-scale devices deployed at the edge of the mobile network for obtaining a high-quality user experience. However, since the environment of MEC is dynamic, most existing works focusing on task graph offloading, which rely heavily on expert knowledge or accurate analytical models, fail to fully adapt to such environmental changes, resulting in the reduction of user experience. This paper investigates the task graph offloading in MEC, considering the time-varying computation capabilities of edge computing devices. To adapt to environmental changes, we model the task graph scheduling for computation offloading as a Markov Decision Process (MDP). Then, we design a deep reinforcement learning algorithm (SATA-DRL) to learn the task scheduling strategy from the interaction with the environment, to improve user experience. Extensive simulations validate that SATA-DRL is superior to existing strategies in terms of reducing average makespan and deadline violation.Comment: 13 figure

Deep Reinforcement Learning for Computation Offloading in Mobile Edge Computing

As 5G-networks are deployed worldwide, mobile edge computing (MEC) has been developed to help alleviate resource-intensive computations from an application. Here, IoT devices can offload their computation to an MEC server and receive the computed result. This offloading scheme can be viewed as an optimization problem, where the complexity quickly increases when more devices join the system. In this thesis, we solve the optimization problem and introduce different strategies that are compared to the optimal solution. The strategies implemented are full local computing, full offload to an MEC server, random search, optimal solution, Q- learning, and a deep Q-network (DQN). The main objective for each strategy is to minimize the total cost of the system, where the cost is a combination of energy consumption and delay. However, as the number of devices in the system increases, the results view numerous challenges. This thesis shows that the performance of random search, Q-learning, and DQN strategies are very close to the optimal solution for up to 20 devices. However, the results show generally poor performance for the strategies that can address more than 20 devices. In the end, we further discuss the performance and convergence of a DQN in MEC.Masteroppgave i informatikkINF399MAMN-INFMAMN-PRO

Reconfigurable intelligent surface for low-latency edge computing in 6G

Edge computing, as one of the key technologies in 6G networks, establishes a distributed computing environment by deploying computation and storage resources in proximity to end users. However, the dense deployment of base stations, cellular dead zones, and high dynamics of mobile devices may cause serious interference issues and weak signal propagation, which will severely affect the transmission efficiency of edge computing and cannot support low-latency applications and services. Reconfigurable intelligent surface (RIS) is a new technology that can enhance the spectral efficiency and suppress interference of wireless communication by adaptively configuring massive low-cost passive reflecting elements. In this article, we introduce RIS into edge computing to support low-latency applications, where edge computing can alleviate the heavy computation pressure of mobile devices with ubiquitously distributed computing resources, and RIS can enhance the quality of the wireless communication link by intelligently altering the radio propagation environment. To elaborate the effectiveness of RIS for edge computing, we then propose a deep-reinforcement-learning-based computation offloading scheme to minimize the total offloading latency of mobile devices. Numerical results indicate that the RIS-aided scheme can improve wireless communication data rate and reduce task execution latency

AdaMEC: Towards a Context-Adaptive and Dynamically-Combinable DNN Deployment Framework for Mobile Edge Computing

With the rapid development of deep learning, recent research on intelligent and interactive mobile applications (e.g., health monitoring, speech recognition) has attracted extensive attention. And these applications necessitate the mobile edge computing scheme, i.e., offloading partial computation from mobile devices to edge devices for inference acceleration and transmission load reduction. The current practices have relied on collaborative DNN partition and offloading to satisfy the predefined latency requirements, which is intractable to adapt to the dynamic deployment context at runtime. AdaMEC, a context-adaptive and dynamically-combinable DNN deployment framework is proposed to meet these requirements for mobile edge computing, which consists of three novel techniques. First, once-for-all DNN pre-partition divides DNN at the primitive operator level and stores partitioned modules into executable files, defined as pre-partitioned DNN atoms. Second, context-adaptive DNN atom combination and offloading introduces a graph-based decision algorithm to quickly search the suitable combination of atoms and adaptively make the offloading plan under dynamic deployment contexts. Third, runtime latency predictor provides timely latency feedback for DNN deployment considering both DNN configurations and dynamic contexts. Extensive experiments demonstrate that AdaMEC outperforms state-of-the-art baselines in terms of latency reduction by up to 62.14% and average memory saving by 55.21%

Joint Latency-Energy optimization scheme for Offloading in Mobile Edge computing environment based in Deep Reinforcement Learning

With the increasing number of mobile devices (MD), IoT devices, and computation-intensive tasks deployed on these devices, there is a need to increase the efficiency and speed of the deliverable. Due to inadequate resources, it is infeasible to compute all the tasks locally. Similarly, due to time constraints, it is not possible to compute the entire task at a remote site. Edge computing (EC) and cloud computing (CC) play the role of providing the resources to these devices on the fly. But a major drawback is increased delay and energy consumption due to transmission and offloading of computation tasks to these remote systems. There is a need to divide the task for computation at local sites, edge servers, and cloud servers to complete tasks with minimum delay and energy consumption. This paper proposes offloading strategy computation using Multi-Period Deep Deterministic Policy Gradient (MP-DDPG) algorithm based on Reinforcement Learning (RL) to optimize the latency caused and energy consumed. We formulate our problem as a Multi-period Markov Decision Process (MP-MDP). In this paper, we use the two-tier offloading architecture including more than one mobile device (MD), two EC-servers, and one CC-server as computation sites. Further, we also compare our proposed algorithm using one-tier architecture and one edge server with the Deep Deterministic Policy Gradient (DDPG) algorithm with similar architecture