Reinforcing Reachable Routes

This paper studies the evaluation of routing algorithms from the perspective of reachability routing, where the goal is to determine all paths between a sender and a receiver. Reachability routing is becoming relevant with the changing dynamics of the Internet and the emergence of low-bandwidth wireless/ad-hoc networks. We make the case for reinforcement learning as the framework of choice to realize reachability routing, within the confines of the current Internet infrastructure. The setting of the reinforcement learning problem offers several advantages,including loop resolution, multi-path forwarding capability, cost-sensitive routing, and minimizing state overhead, while maintaining the incremental spirit of current backbone routing algorithms. We identify research issues in reinforcement learning applied to the reachability routing problem to achieve a fluid and robust backbone routing framework. This paper also presents the design, implementation and evaluation of a new reachability routing algorithm that uses a model-based approach to achieve cost-sensitive multi-path forwarding; performance assessment of the algorithm in various troublesome topologies shows consistently superior performance over classical reinforcement learning algorithms. The paper is targeted toward practitioners seeking to implement a reachability routing algorithm

Traffic matrix estimation with enhanced origin destination generator algorithm using simulation of real network

The rapid growth of the Internet has made the issue of ensuring reliability and redundancy a big challenge. Studies of these issues using Traffic Engineering and simulation have been extensively done. In Traffic Matrix Estimation (TME), the Origin–Destination Generator algorithm (ODGen) is limited to the number of hops, where the Expectation Maximization (EM) accuracy is 92%. Most studies have not taken into account real traffic parameters and integration of TME models with routing protocols in their simulation models. Also, there is no a comprehensive model consisting of TME, Border Gateway Protocol (BGP) and Hot Potato (HP) routing in the NS-2 network simulator based on real networks. In this research, Integrated Simulated Model (ISM) is introduced consisting of ODGen-HP algorithm and BGP integrated into the NS-2 network simulator. ISM is then used to simulate the infrastructure of a real production network using actual captured traffic data parameters. Validation is then done against the changes in network topology based on packet loss, delay and throughput. Results gave the average error for packet sent by simulated and production networks of 0% and the average error for packet received by simulation and production networks of 3.61%. The network is modelled with a baseline topology where 5 main nodes were connected together, with redundant links for some nodes. The simulations were repeated for link failures, node addition, and node removal. TME used in ISM is based on ODGen, that is optimized with unlimited number of hops, the accuracy of EM increases to 97% and Central Processing Unit complexity is reduced. HP helps in improving the node which experiences a link failure to select shorter distance route to egress router. In the case of a link failure, HP switching time between the links is 0.05 seconds. ISM performance was evaluated by comparing trace file before and after link failure or by adding nodes (up to 32) or removing nodes. The parameters used for comparison are the packets loss, delay and throughput. The ISM error percentage obtained for packets loss is 0.025%, delay 0.013% and throughput 0.003%

Autonomous Traffic Engineering using Deep Reinforcement Learning

The evolution of communication technologies in the past few decades, has led to a huge increase in the complexity and the overall size of telecommunication networks. This phenomenon has increased the need for innovation in the field of Traffic Engineering (TE), as the already existing solutions are not flexible enough to adapt to these changes. With the appearance of 5G technologies, the urgency to revolutionize the field is higher than ever and the softwarization and virtualization of the infrastructure bring new possibilities for TE optimization, namely the possible use of Artificial Intelligence (AI) based methods for Traffic Management. The recent advances in AI have provided model-free optimization methods with algorithms like Deep Reinforcement Learning (DRL) that can be used to optimize traffic distributions in complex and hard to model Network scenarios. This thesis aims to provide a DRL-based solution for TE where an agent is capable of making routing decisions based on the current state of the network, with the goal of balancing the load between the network paths. A DRL agent is developed and trained in two different scenarios where the traffic that already exists in the network is generated randomly or according to a systematic pattern. A simulation environment was developed to train and evaluate the DRL agent.A evolução das tecnologias de comunicação nas útlimas décadas, tem dado origem a um grande aumento na complexidade e no tamanho das redes de telecomunicações. Este fenómeno tem aumentado a necessidade de inovação na área de Traffic Engineering (TE), visto que as soluções já existentes não são flexíveis o suficiente para se adaptarem a estas mudanças. Com a aproximação das tecnologias 5G, a urgência para revolucionar a área está cada vez maior e a softwarização e a virtualização das infraestruturas trazem novas possibilidades para otimizações de TE, nomeadamente o possível uso de métodos baseados em Inteligência Artificial (IA) para gerir o tráfego da rede. Os avanços recentes de IA têm criado métodos de otimização independentes de modelos (model-free), como Deep Reinforcement Learning (DRL) que pode ser usado para otimizar a distribuição de tráfego em cenários de redes complexas e difíceis de modelar. Esta dissertação tem como objetivo implementar uma solução à base de DRL, em que é desenhado um agente que é capaz de tomar decisões de encaminhamento, com o objetivo de gerir o tráfego pelos caminhos da rede. Um agente DRL é treinado em dois cenários diferentes onde o tráfego já existente na rede é gerado aleatóriamente ou de acordo com um padrão pré-definido. Foi criado um ambiente para treinar e para avaliar o agente DRL

Analysis on a mobile agent-based algorithm for network routing and management

Ant routing is a method for network routing in agent technology. Although its effectiveness and efficiency have been demonstrated and reported in the literature, its properties have not yet been well studied. This paper presents some preliminary analysis on an ant algorithm in regard to its population growing property and jumping behavior. Results conclude that as long as the value max, {i/spl Omega//sub j/|} is known, the practitioner is able to design the algorithm parameters, such as the number of agents being created for each request, k, and the maximum allowable number of jumps of an agent, in order to meet the network constraint.John Sum, Hong Shen, Chi-sing Leung, and G. Youn

Modeling Multi-aspect Preferences and Intents for Multi-behavioral Sequential Recommendation

Multi-behavioral sequential recommendation has recently attracted increasing attention. However, existing methods suffer from two major limitations. Firstly, user preferences and intents can be described in fine-grained detail from multiple perspectives; yet, these methods fail to capture their multi-aspect nature. Secondly, user behaviors may contain noises, and most existing methods could not effectively deal with noises. In this paper, we present an attentive recurrent model with multiple projections to capture Multi-Aspect preferences and INTents (MAINT in short). To extract multi-aspect preferences from target behaviors, we propose a multi-aspect projection mechanism for generating multiple preference representations from multiple aspects. To extract multi-aspect intents from multi-typed behaviors, we propose a behavior-enhanced LSTM and a multi-aspect refinement attention mechanism. The attention mechanism can filter out noises and generate multiple intent representations from different aspects. To adaptively fuse user preferences and intents, we propose a multi-aspect gated fusion mechanism. Extensive experiments conducted on real-world datasets have demonstrated the effectiveness of our model

Novel CropdocNet Model for Automated Potato Late Blight Disease Detection from Unmanned Aerial Vehicle-Based Hyperspectral Imagery

The accurate and automated diagnosis of potato late blight disease, one of the most destructive potato diseases, is critical for precision agricultural control and management. Recent advances in remote sensing and deep learning offer the opportunity to address this challenge. This study proposes a novel end-to-end deep learning model (CropdocNet) for accurate and automated late blight disease diagnosis from UAV-based hyperspectral imagery. The proposed method considers the potential disease-specific reflectance radiation variance caused by the canopy’s structural diversity and introduces multiple capsule layers to model the part-to-whole relationship between spectral– spatial features and the target classes to represent the rotation invariance of the target classes in the feature space. We evaluate the proposed method with real UAV-based HSI data under controlled and natural field conditions. The effectiveness of the hierarchical features is quantitatively assessed and compared with the existing representative machine learning/deep learning methods on both testing and independent datasets. The experimental results show that the proposed model significantly improves accuracy when considering the hierarchical structure of spectral–spatial features, with average accuracies of 98.09% for the testing dataset and 95.75% for the independent dataset, respectively