Multi-Agent Distributed Reinforcement Learning for Making Decentralized Offloading Decisions

We formulate computation offloading as a decentralized decision-making problem with autonomous agents. We design an interaction mechanism that incentivizes agents to align private and system goals by balancing between competition and cooperation. The mechanism provably has Nash equilibria with optimal resource allocation in the static case. For a dynamic environment, we propose a novel multi-agent online learning algorithm that learns with partial, delayed and noisy state information, and a reward signal that reduces information need to a great extent. Empirical results confirm that through learning, agents significantly improve both system and individual performance, e.g., 40% offloading failure rate reduction, 32% communication overhead reduction, up to 38% computation resource savings in low contention, 18% utilization increase with reduced load variation in high contention, and improvement in fairness. Results also confirm the algorithm's good convergence and generalization property in significantly different environments

Emerging Edge Computing Technologies for Distributed Internet of Things (IoT) Systems

The ever-increasing growth in the number of connected smart devices and various Internet of Things (IoT) verticals is leading to a crucial challenge of handling massive amount of raw data generated from distributed IoT systems and providing real-time feedback to the end-users. Although existing cloud-computing paradigm has an enormous amount of virtual computing power and storage capacity, it is not suitable for latency-sensitive applications and distributed systems due to the involved latency and its centralized mode of operation. To this end, edge/fog computing has recently emerged as the next generation of computing systems for extending cloud-computing functions to the edges of the network. Despite several benefits of edge computing such as geo-distribution, mobility support and location awareness, various communication and computing related challenges need to be addressed in realizing edge computing technologies for future IoT systems. In this regard, this paper provides a holistic view on the current issues and effective solutions by classifying the emerging technologies in regard to the joint coordination of radio and computing resources, system optimization and intelligent resource management. Furthermore, an optimization framework for edge-IoT systems is proposed to enhance various performance metrics such as throughput, delay, resource utilization and energy consumption. Finally, a Machine Learning (ML) based case study is presented along with some numerical results to illustrate the significance of edge computing.Comment: 16 pages, 4 figures, 2 tables, submitted to IEEE Wireless Communications Magazin

Towards More Efficient 5G Networks via Dynamic Traffic Scheduling

Department of Electrical EngineeringThe 5G communications adopt various advanced technologies such as mobile edge computing and unlicensed band operations, to meet the goal of 5G services such as enhanced Mobile Broadband (eMBB) and Ultra Reliable Low Latency Communications (URLLC). Specifically, by placing the cloud resources at the edge of the radio access network, so-called mobile edge cloud, mobile devices can be served with lower latency compared to traditional remote-cloud based services. In addition, by utilizing unlicensed spectrum, 5G can mitigate the scarce spectrum resources problem thus leading to realize higher throughput services. To enhance user-experienced service quality, however, aforementioned approaches should be more fine-tuned by considering various network performance metrics altogether. For instance, the mechanisms for mobile edge computing, e.g., computation offloading to the edge cloud, should not be optimized in a specific metric's perspective like latency, since actual user satisfaction comes from multi-domain factors including latency, throughput, monetary cost, etc. Moreover, blindly combining unlicensed spectrum resources with licensed ones does not always guarantee the performance enhancement, since it is crucial for unlicensed band operations to achieve peaceful but efficient coexistence with other competing technologies (e.g., Wi-Fi). This dissertation proposes a focused resource management framework for more efficient 5G network operations as follows. First, Quality-of-Experience is adopted to quantify user satisfaction in mobile edge computing, and the optimal transmission scheduling algorithm is derived to maximize user QoE in computation offloading scenarios. Next, regarding unlicensed band operations, two efficient mechanisms are introduced to improve the coexistence performance between LTE-LAA and Wi-Fi networks. In particular, we develop a dynamic energy-detection thresholding algorithm for LTE-LAA so that LTE-LAA devices can detect Wi-Fi frames in a lightweight way. In addition, we propose AI-based network configuration for an LTE-LAA network with which an LTE-LAA operator can fine-tune its coexistence parameters (e.g., CAA threshold) to better protect coexisting Wi-Fi while achieving enhanced performance than the legacy LTE-LAA in the standards. Via extensive evaluations using computer simulations and a USRP-based testbed, we have verified that the proposed framework can enhance the efficiency of 5G.clos