A single-player Monte Carlo tree search method combined with node importance for virtual network embedding

© 2020, Institut Mines-Télécom and Springer Nature Switzerland AG. As a critical technology in network virtualization, virtual network embedding (VNE) focuses on how to allocate physical resources to virtual network requests efficiently. Because the VNE problem is NP-hard, most of the existing approaches are heuristic-based algorithms that tend to converge to a local optimal solution and have a low performance. In this paper, we propose an algorithm that combines the basic Monte Carlo tree search (MCTS) method with node importance to apply domain-specific knowledge. For a virtual network request, we first model the embedding process as a finite Markov decision process (MDP), where each virtual node is embedded in one state in the order of node importance that we design. The shortest-path algorithm is then applied to embed links in the terminal state and return the cost as a part of the reward. Due to the reward delay mechanism of the MDP, the result of link mapping can affect the action selected in the previous node mapping stage, coordinating the two embedding stages. With node importance, domain-specific knowledge can be used in the Expansion and Simulation stages of MCTS to speed up the search and estimate the simulation value more accurately. The experimental results show that, compared with the existing classic algorithms, our proposed algorithm can improve the performance of VNE in terms of the average physical node utilization ratio, acceptance ratio, and long-term revenue to cost ratio

Survivable Virtual Network Embedding in Transport Networks

Network Virtualization (NV) is perceived as an enabling technology for the future Internet and the 5th Generation (5G) of mobile networks. It is becoming increasingly difficult to keep up with emerging applications’ Quality of Service (QoS) requirements in an ossified Internet. NV addresses the current Internet’s ossification problem by allowing the co-existence of multiple Virtual Networks (VNs), each customized to a specific purpose on the shared Internet. NV also facilitates a new business model, namely, Network-as-a-Service (NaaS), which provides a separation between applications and services, and the networks supporting them. 5G mobile network operators have adopted the NaaS model to partition their physical network resources into multiple VNs (also called network slices) and lease them to service providers. Service providers use the leased VNs to offer customized services satisfying specific QoS requirements without any investment in deploying and managing a physical network infrastructure. The benefits of NV come at additional resource management challenges. A fundamental problem in NV is to efficiently map the virtual nodes and virtual links of a VN to physical nodes and paths, respectively, known as the Virtual Network Embedding (VNE) problem. A VNE that can survive physical resource failures is known as the survivable VNE (SVNE) problem, and has received significant attention recently. In this thesis, we address variants of the SVNE problem with different bandwidth and reliability requirements for transport networks. Specifically, the thesis includes four main contributions. First, a connectivity-aware VNE approach that ensures VN connectivity without bandwidth guarantee in the face of multiple link failures. Second, a joint spare capacity allocation and VNE scheme that provides bandwidth guarantee against link failures by augmenting VNs with necessary spare capacity. Third, a generalized recovery mechanism to re-embed the VNs that are impacted by a physical node failure. Fourth, a reliable VNE scheme with dedicated protection that allows tuning of available bandwidth of a VN during a physical link failure. We show the effectiveness of the proposed SVNE schemes through extensive simulations. We believe that the thesis can set the stage for further research specially in the area of automated failure management for next generation networks

Self Adaptive Reinforcement Learning for High-Dimensional Stochastic Systems with Application to Robotic Control

A long standing goal in the field of artificial intelligence (AI) is to develop agents that can perceive richer problem space and effortlessly plan their activity in minimal duration. Several strides have been made towards this goal over the last few years due to simultaneous advances in compute power, optimized algorithms, and most importantly evident success of AI based machines in nearly every discipline. The progress has been especially rapid in area of reinforcement learning (RL) where computers can now plan-ahead their activities and outperform their human rivals in complex problem domains like chess or Go game. However, despite encouraging progress, most of the advances in RL-based planning still take place in deterministic context (e.g. constant grid size, known action sets, etc.) which does not adapts well to stochastic variations in problem domain. In this dissertation we develop techniques that enable self-adaptation of agent\u27s behavioral policy when exposed to variations in problem domain. In particular, first we introduce an initial model that loosely realizes problem domain\u27s characteristics. The domain characteristics are embedded into a common multi-modal embedding space set. The embedding space set then allows us to identify initial beliefs and establish prior distributions without being constrained to only finite collection of agent\u27s state-action-reward experiences to choose from. We describe a learning technique that adapts to variations in problem domain by retaining only salient features of preceding domains, and inferring posterior for newly introduced variation as direct perturbation to aggregated priors. Besides having theoretical guarantees, we demonstrate end-to-end solution by establishing FPGA-based recurrent neural network, that can change its synaptic architecture temporally, thus eliminating the need of maintaining dual networks. We argue that our hardware based neural implementation has practical benefits, due to the fact it only uses sparse network architecture and multiplex it on circuit level to exhibit recurrence, which can reduce inference latency on circuit-level, while maintaining equivalence to dense neural architecture

Enabling Technology in Optical Fiber Communications: From Device, System to Networking

This book explores the enabling technology in optical fiber communications. It focuses on the state-of-the-art advances from fundamental theories, devices, and subsystems to networking applications as well as future perspectives of optical fiber communications. The topics cover include integrated photonics, fiber optics, fiber and free-space optical communications, and optical networking

Scalable Orchestration of Service Function Chains in NFV-Enabled Networks: A Federated Reinforcement Learning Approach

This is the author accepted manuscript. The final version is available from IEEE via the DOI in this recordNetwork function virtualization (NFV) is critical to the scalability and flexibility of various network services in the form of service function chains (SFCs), which refer to a set of Virtual Network Functions (VNFs) chained in a specific order. However, the NFV performance is hard to fulfill the ever-increasing requirements of network services mainly due to the static orchestrations of SFCs. To tackle this issue, a novel Scalable SFC Orchestration (SSCO) scheme is proposed in this paper for NFV-enabled networks via federated reinforcement learning. SSCO has three remarkable characteristics distinguishing from the previous work: (1) A federated-learning-based framework is designed to train a global learning model, with time-variant local model explorations, for scalable SFC orchestration, while avoiding data sharing among stakeholders; (2) SSCO allows for parameter update among local clients and the cloud server just at the first and last epochs of each episode to ensure that distributed clients can make model optimization at a low communication cost; (3) SSCO introduces an efficient deep reinforcement learning (DRL) approach, with the local learning knowledge of available resources and instantiation cost, to map VNFs into networks flexibly. Furthermore, a loss-weight-based mechanism is proposed to generate and exploit reference samples in replay buffers for future training, avoiding the strong relevance of samples. Simulation results obtained from different working scenarios demonstrate that SSCO can significantly reduce placement errors and improve resource utilization ratio to place time-variant VNFs compared with the state-of-the-art mechanisms. Furthermore, the results show that the proposed approach can achieve desirable scalability

Management of Spectral Resources in Elastic Optical Networks

Recent developments in the area of mobile technologies, data center networks, cloud computing and social networks have triggered the growth of a wide range of network applications. The data rate of these applications also vary from a few megabits per second (Mbps) to several Gigabits per second (Gbps), thereby increasing the burden on the Inter- net. To support this growth in Internet data traffic, one foremost solution is to utilize the advancements in optical networks. With technology such as wavelength division multiplexing (WDM) networks, bandwidth upto 100 Gbps can be exploited from the optical fiber in an energy efficient manner. However, WDM networks are not efficient when the traffic demands vary frequently. Elastic Optical Networks (EONs) or Spectrum Sliced Elastic Optical Path Networks (SLICE) or Flex-Grid has been recently proposed as a long-term solution to handle the ever-increasing data traffic and the diverse demand range. EONs provide abundant bandwidth by managing the spectrum resources as fine-granular orthogonal sub-carriers that makes it suitable to accommodate varying traffic demands. However, the Routing and Spectrum Allocation (RSA) algorithm in EONs has to follow additional constraints while allocating sub-carriers to demands. These constraints increase the complexity of RSA in EONs and also, make EONs prone to the fragmentation of spectral resources, thereby decreasing the spectral efficiency. The major objective of this dissertation is to study the problem of spectrum allocation in EONs under various network conditions. With this objective, this dissertation presents the author\u27s study and research on multiple aspects of spectrum allocation in EONs: how to allocate sub-carriers to the traffic demands, how to accommodate traffic demands that varies with time, how to minimize the fragmentation of spectral resources and how to efficiently integrate the predictability of user demands for spectrum assignment. Another important contribution of this dissertation is the application of EONs as one of the substrate technologies for network virtualization

Journal of Telecommunications and Information Technology, 2011, nr 4

kwartalni