DHRL-FNMR: An Intelligent Multicast Routing Approach Based on Deep Hierarchical Reinforcement Learning in SDN

The optimal multicast tree problem in the Software-Defined Networking (SDN) multicast routing is an NP-hard combinatorial optimization problem. Although existing SDN intelligent solution methods, which are based on deep reinforcement learning, can dynamically adapt to complex network link state changes, these methods are plagued by problems such as redundant branches, large action space, and slow agent convergence. In this paper, an SDN intelligent multicast routing algorithm based on deep hierarchical reinforcement learning is proposed to circumvent the aforementioned problems. First, the multicast tree construction problem is decomposed into two sub-problems: the fork node selection problem and the construction of the optimal path from the fork node to the destination node. Second, based on the information characteristics of SDN global network perception, the multicast tree state matrix, link bandwidth matrix, link delay matrix, link packet loss rate matrix, and sub-goal matrix are designed as the state space of intrinsic and meta controllers. Then, in order to mitigate the excessive action space, our approach constructs different action spaces at the upper and lower levels. The meta-controller generates an action space using network nodes to select the fork node, and the intrinsic controller uses the adjacent edges of the current node as its action space, thus implementing four different action selection strategies in the construction of the multicast tree. To facilitate the intelligent agent in constructing the optimal multicast tree with greater speed, we developed alternative reward strategies that distinguish between single-step node actions and multi-step actions towards multiple destination nodes

Joint QoS multicast routing and channel assignment in multiradio multichannel wireless mesh networks using intelligent computational methods

Copyright @ 2010 Elsevier B.V. All rights reserved.In this paper, the quality of service multicast routing and channel assignment (QoS-MRCA) problem is investigated. It is proved to be a NP-hard problem. Previous work separates the multicast tree construction from the channel assignment. Therefore they bear severe drawback, that is, channel assignment cannot work well with the determined multicast tree. In this paper, we integrate them together and solve it by intelligent computational methods. First, we develop a unified framework which consists of the problem formulation, the solution representation, the fitness function, and the channel assignment algorithm. Then, we propose three separate algorithms based on three representative intelligent computational methods (i.e., genetic algorithm, simulated annealing, and tabu search). These three algorithms aim to search minimum-interference multicast trees which also satisfy the end-to-end delay constraint and optimize the usage of the scarce radio network resource in wireless mesh networks. To achieve this goal, the optimization techniques based on state of the art genetic algorithm and the techniques to control the annealing process and the tabu search procedure are well developed separately. Simulation results show that the proposed three intelligent computational methods based multicast algorithms all achieve better performance in terms of both the total channel conflict and the tree cost than those comparative references.This work was supported by the Engineering and Physical Sciences Research Council (EPSRC) of UK under Grant EP/E060722/1

Bi-velocity discrete particle swarm optimization and its application to multicast routing problem in communication networks

This paper proposes a novel bi-velocity discrete particle swarm optimization (BVDPSO) approach and extends its application to the NP-complete multicast routing problem (MRP). The main contribution is the extension of PSO from continuous domain to the binary or discrete domain. Firstly, a novel bi-velocity strategy is developed to represent possibilities of each dimension being 1 and 0. This strategy is suitable to describe the binary characteristic of the MRP where 1 stands for a node being selected to construct the multicast tree while 0 stands for being otherwise. Secondly, BVDPSO updates the velocity and position according to the learning mechanism of the original PSO in continuous domain. This maintains the fast convergence speed and global search ability of the original PSO. Experiments are comprehensively conducted on all of the 58 instances with small, medium, and large scales in the OR-library (Operation Research Library). The results confirm that BVDPSO can obtain optimal or near-optimal solutions rapidly as it only needs to generate a few multicast trees. BVDPSO outperforms not only several state-of-the-art and recent heuristic algorithms for the MRP problems, but also algorithms based on GA, ACO, and PSO

QoS multicast tree construction in IP/DWDM optical internet by bio-inspired algorithms

Copyright @ Elsevier Ltd. All rights reserved.In this paper, two bio-inspired Quality of Service (QoS) multicast algorithms are proposed in IP over dense wavelength division multiplexing (DWDM) optical Internet. Given a QoS multicast request and the delay interval required by the application, both algorithms are able to find a flexible QoS-based cost suboptimal routing tree. They first construct the multicast trees based on ant colony optimization and artificial immune algorithm, respectively. Then a dedicated wavelength assignment algorithm is proposed to assign wavelengths to the trees aiming to minimize the delay of the wavelength conversion. In both algorithms, multicast routing and wavelength assignment are integrated into a single process. Therefore, they can find the multicast trees on which the least wavelength conversion delay is achieved. Load balance is also considered in both algorithms. Simulation results show that these two bio-inspired algorithms can construct high performance QoS routing trees for multicast applications in IP/DWDM optical Internet.This work was supported in part ny the Program for New Century Excellent Talents in University, the Engineering and Physical Sciences Research Council (EPSRC) of UK under Grant EP/E060722/1, the National Natural Science Foundation of China under Grant no. 60673159 and 70671020, the National High-Tech Reasearch and Development Plan of China under Grant no. 2007AA041201, and the Specialized Research Fund for the Doctoral Program of Higher Education under Grant no. 20070145017

Minimum-cost multicast over coded packet networks

We consider the problem of establishing minimum-cost multicast connections over coded packet networks, i.e., packet networks where the contents of outgoing packets are arbitrary, causal functions of the contents of received packets. We consider both wireline and wireless packet networks as well as both static multicast (where membership of the multicast group remains constant for the duration of the connection) and dynamic multicast (where membership of the multicast group changes in time, with nodes joining and leaving the group). For static multicast, we reduce the problem to a polynomial-time solvable optimization problem, and we present decentralized algorithms for solving it. These algorithms, when coupled with existing decentralized schemes for constructing network codes, yield a fully decentralized approach for achieving minimum-cost multicast. By contrast, establishing minimum-cost static multicast connections over routed packet networks is a very difficult problem even using centralized computation, except in the special cases of unicast and broadcast connections. For dynamic multicast, we reduce the problem to a dynamic programming problem and apply the theory of dynamic programming to suggest how it may be solved

A multipopulation parallel genetic simulated annealing based QoS routing and wavelength assignment integration algorithm for multicast in optical networks

Copyright @ 2008 Elsevier B.V. All rights reserved.In this paper, we propose an integrated Quality of Service (QoS) routing algorithm for optical networks. Given a QoS multicast request and the delay interval specified by users, the proposed algorithm can find a flexible-QoS-based cost suboptimal routing tree. The algorithm first constructs the multicast tree based on the multipopulation parallel genetic simulated annealing algorithm, and then assigns wavelengths to the tree based on the wavelength graph. In the algorithm, routing and wavelength assignment are integrated into a single process. For routing, the objective is to find a cost suboptimal multicast tree. For wavelength assignment, the objective is to minimize the delay of the multicast tree, which is achieved by minimizing the number of wavelength conversion. Thus both the cost of multicast tree and the user QoS satisfaction degree can approach the optimal. Our algorithm also considers load balance. Simulation results show that the proposed algorithm is feasible and effective. We also discuss the practical realization mechanisms of the algorithm.This work was supported in part by the Engineering and Physical Sciences Research Council (EPSRC) of UK under Grant EP/E060722/1, the National Natural Science Foundation of China under Grant nos. 60673159 and 70671020, the National High-Tech Research and Development Plan of China under Grant no. 2006AA01Z214, Program for New Century Excellent Talents in University, and the Key Project of Chinese Ministry of Education under Grant no. 108040

Joint multicast routing and channel assignment in multiradio multichannel wireless mesh networks using simulated annealing

This is the post-print version of the article - Copyright @ 2008 Springer-VerlagThis paper proposes a simulated annealing (SA) algorithm based optimization approach to search a minimum-interference multicast tree which satisfies the end-to-end delay constraint and optimizes the usage of the scarce radio network resource in wireless mesh networks. In the proposed SA multicast algorithm, the path-oriented encoding method is adopted and each candidate solution is represented by a tree data structure (i.e., a set of paths). Since we anticipate the multicast trees on which the minimum-interference channel assignment can be produced, a fitness function that returns the total channel conflict is devised. The techniques for controlling the annealing process are well developed. A simple yet effective channel assignment algorithm is proposed to reduce the channel conflict. Simulation results show that the proposed SA based multicast algorithm can produce the multicast trees which have better performance in terms of both the total channel conflict and the tree cost than that of a well known multicast algorithm in wireless mesh networks.This work was supported by the Engineering and Physical Sciences Research Council (EPSRC) of UK under Grant EP/E060722/1

A genetic-inspired joint multicast routing and channel assignment algorithm in wireless mesh networks

Copyright @ 2008 IEEEThis paper proposes a genetic algorithm (GA) based optimization approach to search a minimum-interference multicast tree which satis¯es the end-to-end delay constraint and optimizes the usage of the scarce radio network resource in wireless mesh networks. The path-oriented en- coding method is used and each chromosome is represented by a tree data structure (i.e., a set of paths). Since we expect the multicast trees on which the minimum-interference channel assignment can be produced, a fitness function that returns the total channel conflict is devised. Crossover and mutation are well designed to adapt to the tree structure. A simple yet effective channel assignment algorithm is proposed to reduce the channel conflict. Simulation results show that the proposed GA based multicast algorithm achieves better performance in terms of both the total channel conflict and the tree cost than that of a well known algorithm