A multiadaptive sampling technique for cost-effective network measurements

The deployment of efficient measurement solutions to assist network management tasks without interfering with normal network operation assumes a prominent role in today’s high-speed networks attending to the huge amounts of traffic involved. From a myriad of proposals for traffic measurement, sampling techniques are particularly relevant contributing effectively for this purpose as only a subset of the overall traffic volume is handled for processing, preserving ideally the correct estimation of network statistical behavior. In this context, this paper proposes MuST – a multiadaptive sampling technique based on linear prediction, aiming at reducing significantly the measurement overhead and still assuring that traffic samples reflect the statistical characteristics of the global network traffic under analysis. Conversely to current sampling techniques, MuST is a multi and self-adaptive technique as both the sample size and interval between samples are self-adjustable parameters according to the ongoing network activity and the accuracy of prediction achieved. The tests carried out demonstrate that the proposed sampling technique is able to achieve accurate network estimations with reduced overhead, using throughput as reference parameter. The evaluation results, obtained resorting to real traffic traces representing wired and wireless aggregated traffic scenarios and actual network services, prove that the simplicity, flexibility and self-adaptability of the proposed technique can be successfully explored to improve network measurements efficiency over distinct traffic conditions. For optimization purposes, this paper also includes a study of the impact of varying the order of prediction, i.e., of considering different degrees of past memory in the self-adaptive estimation mechanism. The significance of the obtained results is demonstrated through statistical benchmarking.Fundação para a Ciência e a Tecnologia (FCT

Master of Science

thesisDummynet is a link emulator that can be used by itself, as well as integrated within testbeds such as Emulab. Despite its popularity in the research community, Dummynet still lacks the ability to precisely emulate certain real network effects. In particular, it has no support for packet reordering. Since reordering is a common and prevalent network phe- nomenon just like packet loss or delay, it cannot be ignored when implementing emulators if we want to provide realistic emulation. It has been observed that networks suffer from reordering caused by packet striping, retransmissions, load balancing, multipath forwarding, etc. This has significant nega- tive effects on the performance of both Transmission Control Protocol (TCP) and User Datagram Protocol (UDP). With the increase in prevalence of real-time streaming UDP applications such as video conferencing and Internet Protocol Television (IPTV), it has become important to focus on this problem which affects the performance of all these applications. Research into models and tools to diagnose and understand reordering requires that a sophisticated metric be used to describe it. So, in this thesis, I make two contributions: improving the realism of traffic shaping in Dummynet emulator by adding support for emulation of reordering, and an algorithm, a max-flow solver, that generates reordered sequences to be used by Dummynet, from a sophisticated reordering metric called Reorder Density (RD). My implementation enables the user to specify the desired amount of reordering in a metric, such as RD (or even others), and have Dummynet generate traffic that is reordered according to the input metric's value. This is accomplished within Dummynet by the use of a newly implemented scheduler. I conclude my thesis with an evaluation using real and software generated network traces to show that the algorithm is scalable and the implementation works correctly. Also, a datapath evaluation to show that my modifications to Dummynet do not result in any unnecessary increase in emulation running time is included

Multipoint passive monitoring in packet networks

Traffic monitoring is essential to manage large networks and validate Service Level Agreements. Passive monitoring is particularly valuable to promptly identify transient fault episodes and react in a timely manner. This paper proposes a novel, non-invasive and flexible method to passively monitor large backbone networks. By using only packet counters, commonly available on existing hardware, we can accurately measure packet losses, in different segments of the network, affecting only specific flows. We can monitor not only end-to-end flows, but any generic flow with packets following several different paths in the network (multipoint flows). We also sketch a possible extension of the method to measure average one-way delay for multipoint flows, provided that the measurement points are synchronized. Through various experiments we show that the method is effective and enables easy zooming in on the cause packet losses. Moreover, the method can scale to very large networks with a very low overhead on the data plane and the management plane

Packet level measurement over wireless access

PhDPerformance Measurement of the IP packet networks mainly comprise of monitoring the network performance in terms of packet losses and delays. If used appropriately, these network parameters (i.e. delay, loss and bandwidth etc) can indicate the performance status of the network and they can be used in fault and performance monitoring, network provisioning, and traffic engineering. Globally, there is a growing need for accurate network measurement to support the commercial use of IP networks. In wireless networks, transmission losses and communication delays strongly affect the performance of the network. Compared to wired networks, wireless networks experience higher levels of data dropouts, and corruption due to issues of channel fading, noise, interference and mobility. Performance monitoring is a vital element in the commercial future of broadband packet networking and the ability to guarantee quality of service in such networks is implicit in Service Level Agreements. Active measurements are performed by injecting probes, and this is widely used to determine the end to end performance. End to end delay in wired networks has been extensively investigated, and in this thesis we report on the accuracy achieved by probing for end to end delay over a wireless scenario. We have compared two probing techniques i.e. Periodic and Poisson probing, and estimated the absolute error for both. The simulations have been performed for single hop and multi- hop wireless networks. In addition to end to end latency, Active measurements have also been performed for packet loss rate. The simulation based analysis has been tried under different traffic scenarios using Poisson Traffic Models. We have sampled the user traffic using Periodic probing at different rates for single hop and multiple hop wireless scenarios. 5 Active probing becomes critical at higher values of load forcing the network to saturation much earlier. We have evaluated the impact of monitoring overheads on the user traffic, and show that even small amount of probing overhead in a wireless medium can cause large degradation in network performance. Although probing at high rate provides a good estimation of delay distribution of user traffic with large variance yet there is a critical tradeoff between the accuracy of measurement and the packet probing overhead. Our results suggest that active probing is highly affected by probe size, rate, pattern, traffic load, and nature of shared medium, available bandwidth and the burstiness of the traffic

Open source traffic analyzer

Proper traffic analysis is crucial for the development of network systems, services and protocols. Traffic analysis equipment is often based on costly dedicated hardware, and uses proprietary software for traffic generation and analysis. The recent advances in open source packet processing, with the potential of generating and receiving packets using a regular Linux computer at 10 Gb/s speed, opens up very interesting possibilities in terms of implementing a traffic analysis system based on open-source Linux. The pktgen software package for Linux is a popular tool in the networking community for generating traffic loads for network experiments. Pktgen is a high-speed packet generator, running in the Linux kernel very close to the hardware, thereby making it possible to generate packets with very little processing overhead. The packet generation can be controlled through a user interface with respect to packet size, IP and MAC addresses, port numbers, inter-packet delay, and so on. Pktgen was originally designed with the main goal of generating packets at very high rate. However, when it comes to support for traffic analysis, pktgen has several limitations. One of the most important characteristics of a packet generator is the ability to generate traffic at a specified rate. Pktgen can only do this indirectly, by inserting delays between packets. Moreover, the timer granularity prevents precise control of the transmission rate, something which severely reduces pktgen's usefulness as an analysis tool. Furthermore, pktgen lacks support for receiveside analysis and statistics generation. This is a key issue in order to convert pktgen into a useful network analyser tool. In this paper, improvements to pktgen are proposed, designed, implemented and evaluated, with the goal of evolving pktgen into a complete and efficient network analysis tool. The rate control is significantly improved, increasing the resolution and improving the usability by making it possible to specify exactly the sending rate. A receive-side tool is designed and implemented with support for measurement of number of packets, throughput, inter-arrival time, jitter and latency. The design of the receiver takes advantage of SMP systems and new features on modern network cards, in particular support for multiple receive queues and CPU scheduling. This makes it possible to use multiple CPUs to parallelize the work, improving the overall capacity of the traffic analyser. A significant part of the work has been spent on investigating low-level details of Linux networking. From this work we draw some general conclusions related to high speed packet processing in SMP systems. In particular, we study how the packet processing capacity per CPU depends on the number of CPUs. This work consists of minimal set of kernel patches to pktgen

Doctor of Philosophy

dissertationNetwork emulation has become an indispensable tool for the conduct of research in networking and distributed systems. It offers more realism than simulation and more control and repeatability than experimentation on a live network. However, emulation testbeds face a number of challenges, most prominently realism and scale. Because emulation allows the creation of arbitrary networks exhibiting a wide range of conditions, there is no guarantee that emulated topologies reflect real networks; the burden of selecting parameters to create a realistic environment is on the experimenter. While there are a number of techniques for measuring the end-to-end properties of real networks, directly importing such properties into an emulation has been a challenge. Similarly, while there exist numerous models for creating realistic network topologies, the lack of addresses on these generated topologies has been a barrier to using them in emulators. Once an experimenter obtains a suitable topology, that topology must be mapped onto the physical resources of the testbed so that it can be instantiated. A number of restrictions make this an interesting problem: testbeds typically have heterogeneous hardware, scarce resources which must be conserved, and bottlenecks that must not be overused. User requests for particular types of nodes or links must also be met. In light of these constraints, the network testbed mapping problem is NP-hard. Though the complexity of the problem increases rapidly with the size of the experimenter's topology and the size of the physical network, the runtime of the mapper must not; long mapping times can hinder the usability of the testbed. This dissertation makes three contributions towards improving realism and scale in emulation testbeds. First, it meets the need for realistic network conditions by creating Flexlab, a hybrid environment that couples an emulation testbed with a live-network testbed, inheriting strengths from each. Second, it attends to the need for realistic topologies by presenting a set of algorithms for automatically annotating generated topologies with realistic IP addresses. Third, it presents a mapper, assign, that is capable of assigning experimenters' requested topologies to testbeds' physical resources in a manner that scales well enough to handle large environments

Soluções para obtenção e análise de medidas QoS em terminais móveis IP

Mestrado em Engenharia Electrónica e TelecomunicaçõesO aumento do número e complexidade dos serviços, aliado a um novo tipo de clientes cada vez mais exigente, obriga a uma gestão ponderada dos recursos de modo a garantir uma QoS mínima ao utilizador. O facto de, por vezes, as soluções de monitorização de QoS fim a fim não serem suficientes para fornecer as indicações de QoS necessárias, principalmente sob o ponto de vista do utilizador final, fez com que fosse necessário procurar técnicas alternativas capazes de obter essa informação. Nesta dissertação é apresentada uma solução para monitorizar e extrair as principais medidas de QoS associadas à ligação existente entre terminais móveis IP e os respectivos routers de acesso. A informação será obtida através da análise das camadas de rede e transporte, não descartando porém outras camadas de nível inferior e superior. Essa informação será utilizada na estimativa de um valor descritivo da qualidade actual da ligação. Serão apresentados detalhes de implementação, assim como cenários de teste, resultados e principais conclusões. Com este trabalho foi possível obter estimativas de diversos parâmetros importantes para a determinação da QoS e comprovar a viabilidade da solução aqui apresentada. ABSTRACT: The increase of the number and complexity of services allied to a new type of more demanding customers compels to a weighed resources management in order to guarantee a minimum QoS to the user. The fact that, sometimes end to end solutions are not enough to supply the necessary QoS indications, mainly under the final user point of view, makes it necessary to look for alternative techniques capable to get this information. In this dissertation it is presented a solution to monitor and extract the main QoS measurements related with the existent connection between the mobile IP terminal and the access router. The information will be obtained through the analysis of network and transport layers without discarding other layers of lower or higher level. That information will be used in the estimation of a descriptive value for the current link quality. Implementation details will be presented as well as test scenarios, results, and main conclusions. With this work, it was possible to obtain estimations for different parameters important to QoS determination and to prove the viability of the solution herein presented