Statistical analysis of network traffic for anomaly detection and quality of service provisioning

Network-wide traffic analysis and monitoring in large-scale networks is a challenging and expensive task. In this thesis work we have proposed to analyze the traffic of a large-scale IP network from aggregated traffic measurements, reducing measurement overheads and simplifying implementation issues. We have provided contributions in three different networking fields related to network-wide traffic analysis and monitoring in large-scale IP networks. The first contribution regards Traffic Matrix (TM) modeling and estimation, where we have proposed new statistical models and new estimation methods to analyze the Origin-Destination (OD) flows of a large-scale TM from easily available link traffic measurements. The second contribution regards the detection and localization of volume anomalies in the TM, where we have introduced novel methods with solid optimality properties that outperform current well-known techniques for network-wide anomaly detection proposed so far in the literature. The last contribution regards the optimization of the routing configuration in large-scale IP networks, particularly when the traffic is highly variable and difficult to predict. Using the notions of Robust Routing Optimization we have proposed new approaches for Quality of Service provisioning under highly variable and uncertain traffic scenarios. In order to provide strong evidence on the relevance of our contributions, all the methods proposed in this thesis work were validated using real traffic data from different operational networks. Additionally, their performance was compared against well-known works in each field, showing outperforming results in most cases. Taking together the ensemble of developed TM models, the optimal network-wide anomaly detection and localization methods, and the routing optimization algorithms, this thesis work offers a complete solution for network operators to efficiently monitor large-scale IP networks from aggregated traffic measurements and to provide accurate QoS-based performance, even in the event of volume traffic anomalie

Techniques d'ingénierie de trafic dynamique pour l'internet

Network convergence and new applications running on end-hosts result in increasingly variable and unpredictable traffic patterns. By providing origin-destination pairs with several possible paths, Dynamic Load-Balancing (DLB) has proved itself an excellent tool to face this uncertainty. The objective in DLB is to distribute traffic among these paths in real-time so that a certain objective function is optimized. In these dynamic schemes, paths are established a priori and the amount of traffic sent through each of them depends on the current traffic demand and network condition. In this thesis we study and propose various DLB mechanisms, differing in two important aspects. The first difference resides in the assumption, or not, that resources are reserved for each path. The second lies on the objective function, which clearly dictates the performance obtained from the network. However, a performance benchmarking of the possible choices has not been carried out so far. In this sense, for the case in which no reservations are performed, we study and compare several objective functions, including a proposal of ours. We will also propose and study a new distributed algorithm to attain the optimum of these objective functions. Its advantage with respect to previous proposals is its complete self-configuration (i. E. Convergence is guaranteed without any parametrization). Finally, we present the first complete comparative study between DLB and Robust Routing (a fixed routing configuration for all possible traffic demands). In particular, we analyze which scheme is more convenient in each given situation, and highlight some of their respective shortcomings and virtues.Avec la multiplication des services dans un même réseau et les diversités des applications utilisées par les usagers finaux, le trafic transporté est devenu très complexe et dynamique. Le Partage de la Charge Dynamique (PCD) constitue une alternative intéressante pour résoudre cette problématique. Si une paire Source-Destination est connectée par plusieurs chemins, le problème est le suivant : comment distribuer le trafic parmi ces chemins de telle façon qu’une fonction objective soit optimisé. Dans ce cas les chemins sont fixés a priori et la quantité de trafic acheminée sur chaque route est déterminée dynamiquement en fonction de la demande de trafic et de la situation actuelle du réseau. Dans cette thèse nous étudions puis nous proposons plusieurs mécanismes de PCD. Tout d'abord, nous distinguons deux types d’architecture : celles dans lesquelles les ressources sont réservées pour chaque chemin, et celles pour lesquelles aucune réservation n'est effectuée. La simplification faite dans le premier type d’architecture nous permet de proposer l'utilisation d'un nouveau mécanisme pour gérer les chemins. Partant de ce mécanisme, nous définissons un nouvel algorithme de PCD. Concernant la deuxième architecture, nous étudions et comparons plusieurs fonctions objectives. À partir de notre étude, nous proposons un nouvel algorithme distribué permettant d’atteindre l'optimum de ces fonctions objectives. La principale caractéristique de notre algorithme, et son avantage par rapport aux propositions antérieures, est sa capacité d'auto-configuration, dans la mesure où la convergence de l'algorithme est garantie sans aucun besoin de réglage préalable de ses paramètres

Using GRASP and GA to design resilient and cost-effective IP/MPLS networks

The main objective of this thesis is to find good quality solutions for representative instances of the problem of designing a resilient and low cost IP/MPLS network, to be deployed over an existing optical transport network. This research is motivated by two complementary real-world application cases, which comprise the most important commercial and academic networks of Uruguay. To achieve this goal, we performed an exhaustive analysis of existing models and technologies. From all of them we took elements that were contrasted with the particular requirements of our counterparts. We highlight among these requirements, the need of getting solutions transparently implementable over a heterogeneous network environment, which limit us to use widely standardized features of related technologies. We decided to create new models more suitable to fit these needs. These models are intrinsically hard to solve (NP-Hard). Thus we developed metaheuristic based algorithms to find solutions to these real-world instances. Evolutionary Algorithms and Greedy Randomized Adaptive Search Procedures obtained the best results. As it usually happens, real-world planning problems are surrounded by uncertainty. Therefore, we have worked closely with our counterparts to reduce the fuzziness upon data to a set of representative cases. They were combined with different strategies of design to get to scenarios, which were translated into instances of these problems. Finally, the algorithms were fed with this information, and from their outcome we derived our results and conclusions

Análisis estadístico del tráfico de red para la detección de anomalías y la calidad del servicio

Network-wide traffic analysis and monitoring in large-scale networks is a challenging and expensive task. In this thesis work we have proposed to analyze the traffic of a large-scale IP network from aggregated traffic measurements, reducing measurement overheads and simplifying implementation issues. We have provided contributions in three different networking fields related to network-wide traffic analysis and monitoring in large-scale IP networks. The first contribution regards Traffic Matrix (TM) modeling and estimation, where we have proposed new statistical models and new estimation methods to analyze the Origin-Destination (OD) flows of a large-scale TM from easily available link traffic measurements. The second contribution regards the detection and localization of volume anomalies in the TM, where we have introduced novel methods with solid optimality properties that outperform current well-known techniques for network-wide anomaly detection proposed so far in the literature. The last contribution regards the optimization of the routing configuration in large-scale IP networks, particularly when the traffic is highly variable and difficult to predict. Using the notions of Robust Routing Optimization we have proposed new approaches for Quality of Service provisioning under highly variable and uncertain traffic scenarios. In order to provide strong evidence on the relevance of our contributions, all the methods proposed in this thesis work were validated using real traffic data from different operational networks. Additionally, their performance was compared against well-known works in each field, showing outperforming results in most cases. Taking together the ensemble of developed TM models, the optimal network-wide anomaly detection and localization methods, and the routing optimization algorithms, this thesis work offers a complete solution for network operators to efficiently monitor large-scale IP networks from aggregated traffic measurements and to provide accurate QoS-based performance, even in the event of volume traffic anomalies.El monitoreo y el análisis del tráfico de red en redes de gran escala es una tarea costosa y desafiante. En este trabajo de tesis nos hemos propuesto analizar el tráfico de una red IP de gran escala a partir de mediciones de tráfico agregado, reducciendo gastos de monitoreo y simplificando problemas de implementación. Hemos obtenido resultados importantes en tres áreas diferentes relacionadas con el monitoreo y el análisis del tráfico de red en redes IP a gran escala. El primer resultado concierne el modelado y la estimación de la matriz de tráfico (TM), donde hemos propuesto nuevos modelos estadísticos y nuevos métodos de estimación para analizar la flujos Origen-Destino (OD) de una TM a gran escala, a partir de mediciones de volumen en los enlaces de red, fácilmente obtenibles en los sistemas de monitoreo de red de gran escala disponibles en la actualidad. El segundo aporte corresponde con la detección y localización automática de anomalías de volumen en la TM, donde hemos introducido nuevos métodos con sólidas propiedades de optimalidad y cuyo desempeño supera el de las técnicas actualmente propuestas en la literatura para detección de anomalías de red. La última contribución considera la optimización de la configuración del enrutamiento en redes IP a gran escala, especialmente cuando el tráfico en la red es altamente variable y difícil de predecir. Utilizando las nociones de optimización robusta del enrutamiento en la red, hemos propuesto nuevos enfoques para la provisión de calidad de servicio en escenarios donde el tráfico de red es altamente variable e incierto. Con el fin de proporcionar pruebas sólidas sobre la relevancia de nuestras contribuciones, todas los métodos propuestos en este trabajo de tesis han sido evaluados y validados utilizando mediciones de tráfico real en distintas redes operativas. Al mismo tiempo, su desempeño ha sido comparado contra el obtenido por técnicas bien conocidas en cada área, mostrando mejores resultados en la mayoría de los casos. Tomando el conjunto de técnicas desarrolladas respecto del modelado de la TM, la detección y localización óptima de anomalías de red, y los algoritmos de optimización robusta del enrutamineto en la red, este trabajo de tesis ofrece una solución completa para el monitoreo eficiente de redes IP de gran escala a partir de medidas de tráfico agregado, así como también un mecanismo automático para proporcionar niveles de calidad de servicio en caso de anomalías de tráfico