Quality of service routing for real-time traffic

Imperial Users onl

Towards automatic traffic classification and estimation for available bandwidth in IP networks.

Growing rapidly, today's Internet is becoming more difficult to manage. A good understanding of what kind of network traffic classes are consuming network resource as well as how much network resource is available is important for many management tasks like QoS provisioning and traffic engineering. In the light of these objectives, two measurement mechanisms have been explored in this thesis. This thesis explores a new type of traffic classification scheme with automatic and accurate identification capability. First of all, the novel concept of IP flow profile, a unique identifier to the associated traffic class, has been proposed and the relevant model using five IP header based contexts has been presented. Then, this thesis shows that the key statistical features of each context, in the IP flow profile, follows a Gaussian distribution and explores how to use Kohonen Neural Network (KNN) for the purpose of automatically producing IP flow profile map. In order to improve the classification accuracy, this thesis investigates and evaluates the use of PCA for feature selection, which enables the produced patterns to be as tight as possible since tight patterns lead to less overlaps among patterns. In addition, the use of Linear Discriminant Analysis and alternative KNN maps has been investigated as to deal with the overlap issue between produced patterns. The entirety of this process represents a novel addition to the quest for automatic traffic classification in IP networks. This thesis also develops a fast available bandwidth measurement scheme. It firstly addresses the dynamic problem for the one way delay (OWD) trend detection. To deal with this issue, a novel model - asymptotic OWD Comparison (AOC) model for the OWD trend detection has been proposed. Then, three statistical metrics SOT (Sum of Trend), PTC (Positive Trend Checking) and CTC (Complete Trend Comparison) have been proposed to develop the AOC algorithms. To validate the proposed AOC model, an avail-bw estimation tool called Pathpair has been developed and evaluated in the Planetlah environment

Experimenting with commodity 802.11 hardware: overview and future directions

The huge adoption of 802.11 technologies has triggered a vast amount of experimentally-driven research works. These works range from performance analysis to protocol enhancements, including the proposal of novel applications and services. Due to the affordability of the technology, this experimental research is typically based on commercial off-the-shelf (COTS) devices, and, given the rate at which 802.11 releases new standards (which are adopted into new, affordable devices), the field is likely to continue to produce results. In this paper, we review and categorise the most prevalent works carried out with 802.11 COTS devices over the past 15 years, to present a timely snapshot of the areas that have attracted the most attention so far, through a taxonomy that distinguishes between performance studies, enhancements, services, and methodology. In this way, we provide a quick overview of the results achieved by the research community that enables prospective authors to identify potential areas of new research, some of which are discussed after the presentation of the survey.This work has been partly supported by the European Community through the CROWD project (FP7-ICT-318115) and by the Madrid Regional Government through the TIGRE5-CM program (S2013/ICE-2919).Publicad

A Novel Puzzle-Based Framework for Mitigating Distributed Denial of Service Attacks Against Internet Applications

Cryptographic puzzles are promising techniques for mitigating DDoS attacks via decreasing the incoming rate of service eligible requests. However, existing cryptographic puzzle techniques have several shortcomings that make them less appealing as a tool of choice for DDoS defense. These shortcomings include: (1) the lack of accurate models for dynamically determining puzzle hardness; (2) the lack of an efficient and effective counter mechanism for puzzle solution replay attacks; and (3) the wastefulness of the puzzle computations in terms of the clients' computational resources. In this thesis, we provide a puzzle based DDoS defense framework that addresses these shortcomings. Our puzzle framework includes three novel puzzle mechanisms. The first mechanism, called Puzzle+, provides a mathematical model of per-request puzzle hardness. Through extensive experimental study, we show that this model optimizes the effectiveness of puzzle based DDoS mitigation while enabling tight control over the server utilization. In addition, Puzzle+ disables puzzle solution replay attacks by utilizing a novel cache algorithm to detect replays. The second puzzle mechanism, called Productive Puzzles, alleviates the wastefulness of computational puzzles by transforming the puzzle computations into computations of meaningful tasks that provide utility. Our third puzzle mechanism, called Guided Tour Puzzles, eliminates the wasteful puzzle computations all together, and adopts a novel delay-based puzzle construction idea. In addition, it is not affected by the disparity in the computational resources of the client machines that perform the puzzle computations. Through measurement analysis on real network testbeds as well as extensive simulation study, we show that both Productive Puzzles and Guided Tour Puzzles achieve effective mitigation of DDoS attacks while satisfying no wasteful computation requirement. Lastly, we introduce a novel queue management algorithm, called Stochastic Fair Drop Queue (SFDQ), to further strengthen the DDoS protection provided by the puzzle framework. SFDQ is not only effective against DDoS attacks at multiple layers of the protocol stack, it is also simple to configure and deploy. SFDQ is implemented over a novel data structure, called Indexed Linked List, to provide enqueue, dequeue, and remove operations with O(1) time complexity

Monitoring and analysis system for performance troubleshooting in data centers

It was not long ago. On Christmas Eve 2012, a war of troubleshooting began in Amazon data centers. It started at 12:24 PM, with an mistaken deletion of the state data of Amazon Elastic Load Balancing Service (ELB for short), which was not realized at that time. The mistake first led to a local issue that a small number of ELB service APIs were affected. In about six minutes, it evolved into a critical one that EC2 customers were significantly affected. One example was that Netflix, which was using hundreds of Amazon ELB services, was experiencing an extensive streaming service outage when many customers could not watch TV shows or movies on Christmas Eve. It took Amazon engineers 5 hours 42 minutes to find the root cause, the mistaken deletion, and another 15 hours and 32 minutes to fully recover the ELB service. The war ended at 8:15 AM the next day and brought the performance troubleshooting in data centers to world’s attention. As shown in this Amazon ELB case.Troubleshooting runtime performance issues is crucial in time-sensitive multi-tier cloud services because of their stringent end-to-end timing requirements, but it is also notoriously difficult and time consuming. To address the troubleshooting challenge, this dissertation proposes VScope, a flexible monitoring and analysis system for online troubleshooting in data centers. VScope provides primitive operations which data center operators can use to troubleshoot various performance issues. Each operation is essentially a series of monitoring and analysis functions executed on an overlay network. We design a novel software architecture for VScope so that the overlay networks can be generated, executed and terminated automatically, on-demand. From the troubleshooting side, we design novel anomaly detection algorithms and implement them in VScope. By running anomaly detection algorithms in VScope, data center operators are notified when performance anomalies happen. We also design a graph-based guidance approach, called VFocus, which tracks the interactions among hardware and software components in data centers. VFocus provides primitive operations by which operators can analyze the interactions to find out which components are relevant to the performance issue. VScope’s capabilities and performance are evaluated on a testbed with over 1000 virtual machines (VMs). Experimental results show that the VScope runtime negligibly perturbs system and application performance, and requires mere seconds to deploy monitoring and analytics functions on over 1000 nodes. This demonstrates VScope’s ability to support fast operation and online queries against a comprehensive set of application to system/platform level metrics, and a variety of representative analytics functions. When supporting algorithms with high computation complexity, VScope serves as a ‘thin layer’ that occupies no more than 5% of their total latency. Further, by using VFocus, VScope can locate problematic VMs that cannot be found via solely application-level monitoring, and in one of the use cases explored in the dissertation, it operates with levels of perturbation of over 400% less than what is seen for brute-force and most sampling-based approaches. We also validate VFocus with real-world data center traces. The experimental results show that VFocus has troubleshooting accuracy of 83% on average.Ph.D

Techniques for the Analysis of Modern Web Page Traffic using Anonymized TCP/IP Headers

Analysis of traces of network traffic is a methodology that has been widely adopted for studying the Web for several decades. However, due to recent privacy legislation and increasing adoption of traffic encryption, often only anonymized TCP/IP headers are accessible in traffic traces. For traffic traces to remain useful for analysis, techniques must be developed to glean insight using this limited header information. This dissertation evaluates approaches for classifying individual web page downloads — referred to as web page classification — when only anonymized TCP/IP headers are available. The context in which web page classification is defined and evaluated in this dissertation is different from prior traffic classification methods in three ways. First, the impact of diversity in client platforms (browsers, operating systems, device type, and vantage point) on network traffic is explicitly considered. Second, the challenge of overlapping traffic from multiple web pages is explicitly considered and demultiplexing approaches are evaluated (web page segmentation). And lastly, unlike prior work on traffic classification, four orthogonal labeling schemes are considered (genre-based, device-based, navigation-based, and video streaming-based) — these are of value in several web-related applications, including privacy analysis, user behavior modeling, traffic forecasting, and potentially behavioral ad-targeting. We conduct evaluations using large collections of both synthetically generated data, as well as browsing data from real users. Our analysis shows that the client platform choice has a statistically significant impact on web traffic. It also shows that change point detection methods, a new class of segmentation approach, outperform existing idle time-based methods. Overall, this work establishes that web page classification performance can be improved by: (i) incorporating client platform differences in the feature selection and training methodology, and (ii) utilizing better performing web page segmentation approaches. This research increases the overall awareness on the challenges associated with the analysis of modern web traffic. It shows and advocates for considering real-world factors, such as client platform diversity and overlapping traffic from multiple streams, when developing and evaluating traffic analysis techniques.Doctor of Philosoph

Increasing the robustness of networked systems

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2009.Includes bibliographical references (p. 133-143).What popular news do you recall about networked systems? You've probably heard about the several hour failure at Amazon's computing utility that knocked down many startups for several hours, or the attacks that forced the Estonian government web-sites to be inaccessible for several days, or you may have observed inexplicably slow responses or errors from your favorite web site. Needless to say, keeping networked systems robust to attacks and failures is an increasingly significant problem. Why is it hard to keep networked systems robust? We believe that uncontrollable inputs and complex dependencies are the two main reasons. The owner of a web-site has little control on when users arrive; the operator of an ISP has little say in when a fiber gets cut; and the administrator of a campus network is unlikely to know exactly which switches or file-servers may be causing a user's sluggish performance. Despite unpredictable or malicious inputs and complex dependencies we would like a network to self-manage itself, i.e., diagnose its own faults and continue to maintain good performance. This dissertation presents a generic approach to harden networked systems by distinguishing between two scenarios. For systems that need to respond rapidly to unpredictable inputs, we design online solutions that re-optimize resource allocation as inputs change. For systems that need to diagnose the root cause of a problem in the presence of complex subsystem dependencies, we devise techniques to infer these dependencies from packet traces and build functional representations that facilitate reasoning about the most likely causes for faults. We present a few solutions, as examples of this approach, that tackle an important class of network failures. Specifically, we address (1) re-routing traffic around congestion when traffic spikes or links fail in internet service provider networks, (2) protecting websites from denial of service attacks that mimic legitimate users and (3) diagnosing causes of performance problems in enterprises and campus-wide networks. Through a combination of implementations, simulations and deployments, we show that our solutions advance the state-of-the-art.by Srikanth Kandula.Ph.D

Network Traffic Measurements, Applications to Internet Services and Security

The Internet has become along the years a pervasive network interconnecting billions of users and is now playing the role of collector for a multitude of tasks, ranging from professional activities to personal interactions. From a technical standpoint, novel architectures, e.g., cloud-based services and content delivery networks, innovative devices, e.g., smartphones and connected wearables, and security threats, e.g., DDoS attacks, are posing new challenges in understanding network dynamics. In such complex scenario, network measurements play a central role to guide traffic management, improve network design, and evaluate application requirements. In addition, increasing importance is devoted to the quality of experience provided to final users, which requires thorough investigations on both the transport network and the design of Internet services. In this thesis, we stress the importance of users’ centrality by focusing on the traffic they exchange with the network. To do so, we design methodologies complementing passive and active measurements, as well as post-processing techniques belonging to the machine learning and statistics domains. Traffic exchanged by Internet users can be classified in three macro-groups: (i) Outbound, produced by users’ devices and pushed to the network; (ii) unsolicited, part of malicious attacks threatening users’ security; and (iii) inbound, directed to users’ devices and retrieved from remote servers. For each of the above categories, we address specific research topics consisting in the benchmarking of personal cloud storage services, the automatic identification of Internet threats, and the assessment of quality of experience in the Web domain, respectively. Results comprise several contributions in the scope of each research topic. In short, they shed light on (i) the interplay among design choices of cloud storage services, which severely impact the performance provided to end users; (ii) the feasibility of designing a general purpose classifier to detect malicious attacks, without chasing threat specificities; and (iii) the relevance of appropriate means to evaluate the perceived quality of Web pages delivery, strengthening the need of users’ feedbacks for a factual assessment

Combining SOA and BPM Technologies for Cross-System Process Automation

This paper summarizes the results of an industry case study that introduced a cross-system business process automation solution based on a combination of SOA and BPM standard technologies (i.e., BPMN, BPEL, WSDL). Besides discussing major weaknesses of the existing, custom-built, solution and comparing them against experiences with the developed prototype, the paper presents a course of action for transforming the current solution into the proposed solution. This includes a general approach, consisting of four distinct steps, as well as specific action items that are to be performed for every step. The discussion also covers language and tool support and challenges arising from the transformation

DETECTION AND ALLEVIATION OF LAST-MILE WIRELESS LINK BOTTLENECKS

Ph.DDOCTOR OF PHILOSOPH