Passive available bandwidth: Applying self -induced congestion analysis of application-generated traffic

Monitoring end-to-end available bandwidth is critical in helping applications and users efficiently use network resources. Because the performance of distributed systems is intrinsically linked to the performance of the network, applications that have knowledge of the available bandwidth can adapt to changing network conditions and optimize their performance. A well-designed available bandwidth tool should be easily deployable and non-intrusive. While several tools have been created to actively measure the end-to-end available bandwidth of a network path, they require instrumentation at both ends of the path, and the traffic injected by these tools may affect the performance of other applications on the path.;We propose a new passive monitoring system that accurately measures available bandwidth by applying self-induced congestion analysis to traces of application-generated traffic. The Watching Resources from the Edge of the Network (Wren) system transparently provides available bandwidth information to applications without having to modify the applications to make the measurements and with negligible impact on the performance of applications. Wren produces a series of real-time available bandwidth measurements that can be used by applications to adapt their runtime behavior to optimize performance or that can be sent to a central monitoring system for use by other or future applications.;Most active bandwidth tools rely on adjustments to the sending rate of packets to infer the available bandwidth. The major obstacle with using passive kernel-level traces of TCP traffic is that we have no control over the traffic pattern. We demonstrate that there is enough natural variability in the sending rates of TCP traffic that techniques used by active tools can be applied to traces of application-generated traffic to yield accurate available bandwidth measurements.;Wren uses kernel-level instrumentation to collect traces of application traffic and analyzes the traces in the user-level to achieve the necessary accuracy and avoid intrusiveness. We introduce new passive bandwidth algorithms based on the principles of the active tools to measure available bandwidth, investigate the effectiveness of these new algorithms, implement a real-time system capable of efficiently monitoring available bandwidth, and demonstrate that applications can use Wren measurements to adapt their runtime decisions

Why (and How) Networks Should Run Themselves

The proliferation of networked devices, systems, and applications that we depend on every day makes managing networks more important than ever. The increasing security, availability, and performance demands of these applications suggest that these increasingly difficult network management problems be solved in real time, across a complex web of interacting protocols and systems. Alas, just as the importance of network management has increased, the network has grown so complex that it is seemingly unmanageable. In this new era, network management requires a fundamentally new approach. Instead of optimizations based on closed-form analysis of individual protocols, network operators need data-driven, machine-learning-based models of end-to-end and application performance based on high-level policy goals and a holistic view of the underlying components. Instead of anomaly detection algorithms that operate on offline analysis of network traces, operators need classification and detection algorithms that can make real-time, closed-loop decisions. Networks should learn to drive themselves. This paper explores this concept, discussing how we might attain this ambitious goal by more closely coupling measurement with real-time control and by relying on learning for inference and prediction about a networked application or system, as opposed to closed-form analysis of individual protocols

A traffic classification method using machine learning algorithm

Applying concepts of attack investigation in IT industry, this idea has been developed to design a Traffic Classification Method using Data Mining techniques at the intersection of Machine Learning Algorithm, Which will classify the normal and malicious traffic. This classification will help to learn about the unknown attacks faced by IT industry. The notion of traffic classification is not a new concept; plenty of work has been done to classify the network traffic for heterogeneous application nowadays. Existing techniques such as (payload based, port based and statistical based) have their own pros and cons which will be discussed in this literature later, but classification using Machine Learning techniques is still an open field to explore and has provided very promising results up till now

Optimization flow control -- I: Basic algorithm and convergence

We propose an optimization approach to flow control where the objective is to maximize the aggregate source utility over their transmission rates. We view network links and sources as processors of a distributed computation system to solve the dual problem using a gradient projection algorithm. In this system, sources select transmission rates that maximize their own benefits, utility minus bandwidth cost, and network links adjust bandwidth prices to coordinate the sources' decisions. We allow feedback delays to be different, substantial, and time varying, and links and sources to update at different times and with different frequencies. We provide asynchronous distributed algorithms and prove their convergence in a static environment. We present measurements obtained from a preliminary prototype to illustrate the convergence of the algorithm in a slowly time-varying environment. We discuss its fairness property

Network emulation focusing on QoS-Oriented satellite communication

This chapter proposes network emulation basics and a complete case study of QoS-oriented Satellite Communication

Controlo de congestionamento em redes sem fios

Doutoramento em Engenharia ElectrotécnicaCongestion control in wireless networks is an important and open issue. Previous research has proven the poor performance of the Transport Control Protocol (TCP) in such networks. The factors that contribute to the poor performance of TCP in wireless environments concern its unsuitability to identify/detect and react properly to network events, its TCP window based ow control algorithm that is not suitable for the wireless channel, and the congestion collapse due to mobility. New rate based mechanisms have been proposed to mitigate TCP performance in wired and wireless networks. However, these mechanisms also present poor performance, as they lack of suitable bandwidth estimation techniques for multi-hop wireless networks. It is thus important to improve congestion control performance in wireless networks, incorporating components that are suitable for wireless environments. A congestion control scheme which provides an e - cient and fair sharing of the underlying network capacity and available bandwidth among multiple competing applications is crucial to the definition of new e cient and fair congestion control schemes on wireless multi-hop networks. The Thesis is divided in three parts. First, we present a performance evaluation study of several congestion control protocols against TCP, in wireless mesh and ad-hoc networks. The obtained results show that rate based congestion control protocols need an eficient and accurate underlying available bandwidth estimation technique. The second part of the Thesis presents a new link capacity and available bandwidth estimation mechanism denoted as rt-Winf (real time wireless inference). The estimation is performed in real-time and without the need to intrusively inject packets in the network. Simulation results show that rt-Winf obtains the available bandwidth and capacity estimation with accuracy and without introducing overhead trafic in the network. The third part of the Thesis proposes the development of new congestion control mechanisms to address the congestion control problems of wireless networks. These congestion control mechanisms use cross layer information, obtained by rt-Winf, to accurately and eficiently estimate the available bandwidth and the path capacity over a wireless network path. Evaluation of these new proposed mechanisms, through ns-2 simulations, shows that the cooperation between rt-Winf and the congestion control algorithms is able to significantly increase congestion control eficiency and network performance.O controlo de congestionamento continua a ser extremamente importante quando se investiga o desempenho das redes sem fios. Trabalhos anteriores mostram o mau desempenho do Transport Control Proto- col (TCP) em redes sem fios. Os fatores que contribuem para um pior desempenho do TCP nesse tipo de redes s~ao: a sua falta de capacidade para identificar/detetar e reagir adequadamente a eventos da rede; a utilização de um algoritmo de controlo de uxo que não é adequado para o canal sem fios; e o colapso de congestionamento devido á mobilidade. Para colmatar este problemas foram propostos novos mecanismos de controlo de congestionamento baseados na taxa de transmissão. No entanto, estes mecanismos também apresentam um pior desempenho em redes sem fios, já que não utilizam mecanismos adequados para a avaliação da largura de banda disponível. Assim, é importante para melhorar o desempenho do controlo de congestionamento em redes sem fios, incluir componentes que são adequados para esse tipo de ambientes. Um esquema de controlo de congestionamento que permita uma partilha eficiente e justa da capacidade da rede e da largura de banda disponível entre múltiplas aplicações concorrentes é crucial para a definição de novos, eficientes e justos mecanismos de controlo congestionamento para as redes sem fios. A Tese está dividida em três partes. Primeiro, apresentamos um estudo sobre a avaliação de desempenho de vários protocolos de controlo de congestionamento relativamente ao TCP, em redes sem fios em malha e ad-hoc. Os resultados obtidos mostram que os protocolos baseados na taxa de transmissão precisam de uma técnica de avaliação da largura de banda disponível que seja eficiente e precisa . A segunda parte da Tese apresenta um novo mecanismo de avaliação da capacidade da ligação e da largura de banda disponível, designada por rt-Winf (real time wireless inference). A avaliação é realizada em tempo real e sem a necessidade de inserir tráfego na rede. Os resultados obtidos através de simulação e emulação mostram que o rt-Winf obtém com precisão a largura de banda disponível e a capacidade da ligação sem sobrecarregar a rede. A terceira parte da Tese propõe novos mecanismos de controlo de congestionamento em redes sem fios. Estes mecanismos de controlo de congestionamento apresentam um conjunto de caracter ísticas novas para melhorar o seu desempenho, de entre as quais se destaca a utilização da informação de largura de banda disponível obtida pelo rt-Winf. Os resultados da avaliação destes mecanismos, utilizando o simulador ns-2, permitem concluir que a cooperação entre o rt-Winf e os algoritmos de controlo de congestionamento aumenta significativamente o desempenho da rede

Delay-oriented active queue management in TCP/IP networks

PhDInternet-based applications and services are pervading everyday life. Moreover, the growing popularity of real-time, time-critical and mission-critical applications set new challenges to the Internet community. The requirement for reducing response time, and therefore latency control is increasingly emphasized. This thesis seeks to reduce queueing delay through active queue management. While mathematical studies and research simulations reveal that complex trade-off relationships exist among performance indices such as throughput, packet loss ratio and delay, etc., this thesis intends to find an improved active queue management algorithm which emphasizes delay control without trading much on other performance indices such as throughput and packet loss ratio. The thesis observes that in TCP/IP network, packet loss ratio is a major reflection of congestion severity or load. With a properly functioning active queue management algorithm, traffic load will in general push the feedback system to an equilibrium point in terms of packet loss ratio and throughput. On the other hand, queue length is a determinant factor on system delay performance while has only a slight influence on the equilibrium. This observation suggests the possibility of reducing delay while maintaining throughput and packet loss ratio relatively unchanged. The thesis also observes that queue length fluctuation is a reflection of both load changes and natural fluctuation in arriving bit rate. Monitoring queue length fluctuation alone cannot distinguish the difference and identify congestion status; and yet identifying this difference is crucial in finding out situations where average queue size and hence queueing delay can be properly controlled and reasonably reduced. However, many existing active queue management algorithms only monitor queue length, and their control policies are solely based on this measurement. In our studies, our novel finding is that the arriving bit rate distribution of all sources contains information which can be a better indication of congestion status and has a correlation with traffic burstiness. And this thesis develops a simple and scalable way to measure its two most important characteristics, namely the mean ii and the variance of the arriving rate distribution. The measuring mechanism is based on a Zombie List mechanism originally proposed and deployed in Stabilized RED to estimate the number of flows and identify misbehaving flows. This thesis modifies the original zombie list measuring mechanism, makes it capable of measuring additional variables. Based on these additional measurements, this thesis proposes a novel modification to the RED algorithm. It utilizes a robust adaptive mechanism to ensure that the system reaches proper equilibrium operating points in terms of packet loss ratio and queueing delay under various loads. Furthermore, it identifies different congestion status where traffic is less bursty and adapts RED parameters in order to reduce average queue size and hence queueing delay accordingly. Using ns-2 simulation platform, this thesis runs simulations of a single bottleneck link scenario which represents an important and popular application scenario such as home access network or SoHo. Simulation results indicate that there are complex trade-off relationships among throughput, packet loss ratio and delay; and in these relationships delay can be substantially reduced whereas trade-offs on throughput and packet loss ratio are negligible. Simulation results show that our proposed active queue management algorithm can identify circumstances where traffic is less bursty and actively reduce queueing delay with hardly noticeable sacrifice on throughput and packet loss ratio performances. In conclusion, our novel approach enables the application of adaptive techniques to more RED parameters including those affecting queue occupancy and hence queueing delay. The new modification to RED algorithm is a scalable approach and does not introduce additional protocol overhead. In general it brings the benefit of substantially reduced delay at the cost of limited processing overhead and negligible degradation in throughput and packet loss ratio. However, our new algorithm is only tested on responsive flows and a single bottleneck scenario. Its effectiveness on a combination of responsive and non-responsive flows as well as in more complicated network topology scenarios is left for future work