Autonomic Resource Management in Virtual Networks

Virtualization enables the building of multiple virtual networks over a shared substrate. One of the challenges to virtualisation is efficient resource allocation. This problem has been found to be NP hard. Therefore, most approaches to it have not only proposed static solutions, but have also made many assumptions to simplify it. In this paper, we propose a distributed, autonomic and artificial intelligence based solution to resource allocation. Our aim is to obtain self-configuring, selfoptimizing, self-healing and context aware virtual networksComment: Short Paper, 4 Pages, Summer School, PhD Work In Progress Workshop. Scalable and Adaptive Internet Solutions (SAIL). June 201

A Dynamic Service-Migration Mechanism in Edge Cognitive Computing

Driven by the vision of edge computing and the success of rich cognitive services based on artificial intelligence, a new computing paradigm, edge cognitive computing (ECC), is a promising approach that applies cognitive computing at the edge of the network. ECC has the potential to provide the cognition of users and network environmental information, and further to provide elastic cognitive computing services to achieve a higher energy efficiency and a higher Quality of Experience (QoE) compared to edge computing. This paper firstly introduces our architecture of the ECC and then describes its design issues in detail. Moreover, we propose an ECC-based dynamic service migration mechanism to provide an insight into how cognitive computing is combined with edge computing. In order to evaluate the proposed mechanism, a practical platform for dynamic service migration is built up, where the services are migrated based on the behavioral cognition of a mobile user. The experimental results show that the proposed ECC architecture has ultra-low latency and a high user experience, while providing better service to the user, saving computing resources, and achieving a high energy efficiency

Learning-based Prediction, Rendering and Association Optimization for MEC-enabled Wireless Virtual Reality (VR) Network

Wireless-connected Virtual Reality (VR) provides immersive experience for VR users from any-where at anytime. However, providing wireless VR users with seamless connectivity and real-time VR video with high quality is challenging due to its requirements in high Quality of Experience (QoE) and low VR interaction latency under limited computation capability of VR device. To address these issues,we propose a MEC-enabled wireless VR network, where the field of view (FoV) of each VR user can be real-time predicted using Recurrent Neural Network (RNN), and the rendering of VR content is moved from VR device to MEC server with rendering model migration capability. Taking into account the geographical and FoV request correlation, we propose centralized and distributed decoupled Deep Reinforcement Learning (DRL) strategies to maximize the long-term QoE of VR users under the VR interaction latency constraint. Simulation results show that our proposed MEC rendering schemes and DRL algorithms substantially improve the long-term QoE of VR users and reduce the VR interaction latency compared to rendering at VR device

Reinforcement Learning-based Application Autoscaling in the Cloud: A Survey

Reinforcement Learning (RL) has demonstrated a great potential for automatically solving decision-making problems in complex uncertain environments. RL proposes a computational approach that allows learning through interaction in an environment with stochastic behavior, where agents take actions to maximize some cumulative short-term and long-term rewards. Some of the most impressive results have been shown in Game Theory where agents exhibited superhuman performance in games like Go or Starcraft 2, which led to its gradual adoption in many other domains, including Cloud Computing. Therefore, RL appears as a promising approach for Autoscaling in Cloud since it is possible to learn transparent (with no human intervention), dynamic (no static plans), and adaptable (constantly updated) resource management policies to execute applications. These are three important distinctive aspects to consider in comparison with other widely used autoscaling policies that are defined in an ad-hoc way or statically computed as in solutions based on meta-heuristics. Autoscaling exploits the Cloud elasticity to optimize the execution of applications according to given optimization criteria, which demands to decide when and how to scale-up/down computational resources, and how to assign them to the upcoming processing workload. Such actions have to be taken considering that the Cloud is a dynamic and uncertain environment. Motivated by this, many works apply RL to the autoscaling problem in the Cloud. In this work, we survey exhaustively those proposals from major venues, and uniformly compare them based on a set of proposed taxonomies. We also discuss open problems and prospective research in the area.Comment: 40 pages, 9 figure

Applications of Deep Reinforcement Learning in Communications and Networking: A Survey

This paper presents a comprehensive literature review on applications of deep reinforcement learning in communications and networking. Modern networks, e.g., Internet of Things (IoT) and Unmanned Aerial Vehicle (UAV) networks, become more decentralized and autonomous. In such networks, network entities need to make decisions locally to maximize the network performance under uncertainty of network environment. Reinforcement learning has been efficiently used to enable the network entities to obtain the optimal policy including, e.g., decisions or actions, given their states when the state and action spaces are small. However, in complex and large-scale networks, the state and action spaces are usually large, and the reinforcement learning may not be able to find the optimal policy in reasonable time. Therefore, deep reinforcement learning, a combination of reinforcement learning with deep learning, has been developed to overcome the shortcomings. In this survey, we first give a tutorial of deep reinforcement learning from fundamental concepts to advanced models. Then, we review deep reinforcement learning approaches proposed to address emerging issues in communications and networking. The issues include dynamic network access, data rate control, wireless caching, data offloading, network security, and connectivity preservation which are all important to next generation networks such as 5G and beyond. Furthermore, we present applications of deep reinforcement learning for traffic routing, resource sharing, and data collection. Finally, we highlight important challenges, open issues, and future research directions of applying deep reinforcement learning.Comment: 37 pages, 13 figures, 6 tables, 174 reference paper

Performance-Aware Management of Cloud Resources: A Taxonomy and Future Directions

Dynamic nature of the cloud environment has made distributed resource management process a challenge for cloud service providers. The importance of maintaining the quality of service in accordance with customer expectations as well as the highly dynamic nature of cloud-hosted applications add new levels of complexity to the process. Advances to the big data learning approaches have shifted conventional static capacity planning solutions to complex performance-aware resource management methods. It is shown that the process of decision making for resource adjustment is closely related to the behaviour of the system including the utilization of resources and application components. Therefore, a continuous monitoring of system attributes and performance metrics provide the raw data for the analysis of problems affecting the performance of the application. Data analytic methods such as statistical and machine learning approaches offer the required concepts, models and tools to dig into the data, find general rules, patterns and characteristics that define the functionality of the system. Obtained knowledge form the data analysis process helps to find out about the changes in the workloads, faulty components or problems that can cause system performance to degrade. A timely reaction to performance degradations can avoid violations of the service level agreements by performing proper corrective actions including auto-scaling or other resource adjustment solutions. In this paper, we investigate the main requirements and limitations in cloud resource management including a study of the approaches in workload and anomaly analysis in the context of the performance management in the cloud. A taxonomy of the works on this problem is presented which identifies the main approaches in existing researches from data analysis side to resource adjustment techniques

HyMER: A Hybrid Machine Learning Framework for Energy Efficient Routing in SDN

Software-defined networks (SDN) with programmable data plane and machine learning for discovering patterns are utilized in security, traffic classification, quality of services prediction, and network performance, that has increasing research attention. Addressing the significance of energy efficiency in networks, we propose a novel hybrid machine learning-based framework named HyMER that combines the capabilities of SDN and machine learning for traffic-aware energy efficient routing. To the best of our knowledge, HyMER is the first that utilizes a hybrid machine learning solution with supervised and reinforcement learning components for energy efficiency and network performance in SDN. The supervised learning component consists of feature extraction, training, and testing. The reinforcement learning component learns from existing data or from scratch by iteratively interacting with the network environment. The HyMER framework is developed on POX controller and is evaluated on Mininet using real-world topologies and dynamic traffic traces. Experimental results show that the supervised component achieves up to 70% feature size reduction and more than 80\% accuracy in parameter prediction. We demonstrate that combining the supervised and reinforcement methods not only does capture the dynamic change more efficiently but also increases the convergence speed. As compared to state-of-the-art utility based energy saving approaches, HyMER heuristics has shown up to 50% link saving, and also exhibits up to 14.7 watts less power consumption for realistic network topology and traffic traces.Comment: Double column 12 pages, 13 figures, 6 table

Edge Intelligence: The Confluence of Edge Computing and Artificial Intelligence

Along with the rapid developments in communication technologies and the surge in the use of mobile devices, a brand-new computation paradigm, Edge Computing, is surging in popularity. Meanwhile, Artificial Intelligence (AI) applications are thriving with the breakthroughs in deep learning and the many improvements in hardware architectures. Billions of data bytes, generated at the network edge, put massive demands on data processing and structural optimization. Thus, there exists a strong demand to integrate Edge Computing and AI, which gives birth to Edge Intelligence. In this paper, we divide Edge Intelligence into AI for edge (Intelligence-enabled Edge Computing) and AI on edge (Artificial Intelligence on Edge). The former focuses on providing more optimal solutions to key problems in Edge Computing with the help of popular and effective AI technologies while the latter studies how to carry out the entire process of building AI models, i.e., model training and inference, on the edge. This paper provides insights into this new inter-disciplinary field from a broader perspective. It discusses the core concepts and the research road-map, which should provide the necessary background for potential future research initiatives in Edge Intelligence.Comment: 13 pages, 3 figure

Traffic-aware Threshold Adjustment for NFV Scaling using DDPG

Current solutions mostly focus on how to predict traffic, rather than observing traffic characteristics in a specific NFV scenario. So, most of them use a uniform threshold to scale in/out. In real NFV scenario, each VNF may serve the one or more flows, and the characteristics of these flows are completely different, a uniform threshold used in this scenario is not suitable, because each VNF has a distinct processing logic depending on incident network traffic and events. Even if certain VNFs share packet processing functionality such as packet header analysis, the differences in upper-layer processing and implementation can exhibit unique resource usage patterns. We proposes a dynamic threshold scaling mechanism that can tailor thresholds according to each VNF's characteristic. As setting thresholds is a per-VNF task, and requires a deep understanding of workload trends and the diversity of each VNF, so we have added tailor-made features to the traditional dynamic mechanism. Besides, we also reserve resources by predicting workload and add an emergency module to cope with anomaly traffic, that is to say we develop a hybrid scaling policy combining proactive and reactive scaling together. Moreover, the sharp rise of network traffic not only can be caused by large amount of new incoming flows, but also can be induced by the growing of existing flows. If the traffic arises mainly due to the growing of existing flows, then only rerouting new flows can not alleviate the overload quickly and SLAs may be violated \cite{zhang2016co}. The only method to avoid SLA violations is to migrate flows and associated NF internal states quickly and safely from existing instances to new scaled instances, so state migration is an important part of the scaling procedure. We achieved the flow migration in scaling process on openNF to guarantee the accuracy and timeline of scaling

Application Management in Fog Computing Environments: A Taxonomy, Review and Future Directions

The Internet of Things (IoT) paradigm is being rapidly adopted for the creation of smart environments in various domains. The IoT-enabled Cyber-Physical Systems (CPSs) associated with smart city, healthcare, Industry 4.0 and Agtech handle a huge volume of data and require data processing services from different types of applications in real-time. The Cloud-centric execution of IoT applications barely meets such requirements as the Cloud datacentres reside at a multi-hop distance from the IoT devices. \textit{Fog computing}, an extension of Cloud at the edge network, can execute these applications closer to data sources. Thus, Fog computing can improve application service delivery time and resist network congestion. However, the Fog nodes are highly distributed, heterogeneous and most of them are constrained in resources and spatial sharing. Therefore, efficient management of applications is necessary to fully exploit the capabilities of Fog nodes. In this work, we investigate the existing application management strategies in Fog computing and review them in terms of architecture, placement and maintenance. Additionally, we propose a comprehensive taxonomy and highlight the research gaps in Fog-based application management. We also discuss a perspective model and provide future research directions for further improvement of application management in Fog computing