DeviceRadar: Online IoT Device Fingerprinting in ISPs Using Programmable Switches

Device fingerprinting can be used by Internet Service Providers (ISPs) to identify vulnerable IoT devices for early prevention of threats. However, due to the wide deployment of middleboxes in ISP networks, some important data, e.g., 5-tuples and flow statistics, are often obscured, rendering many existing approaches invalid. It is further challenged by the high-speed traffic of hundreds of terabytes per day in ISP networks. This paper proposes DeviceRadar, an online IoT device fingerprinting framework that achieves accurate, real-time processing in ISPs using programmable switches. We innovatively exploit “key packets” as a basis of fingerprints only using packet sizes and directions, which appear periodically while exhibiting differences across different IoT devices. To utilize them, we propose a packet size embedding model to discover the spatial relationships between packets. Meanwhile, we design an algorithm to extract the “key packets” of each device, and propose an approach that jointly considers the spatial relationships and the key packets to produce a neighboring key packet distribution, which can serve as a feature vector for machine learning models for inference. Last, we design a model transformation method and a feature extraction process to deploy the model on a programmable data plane within its constrained arithmetic operations and memory to achieve line-speed processing. Our experiments show that DeviceRadar can achieve state-of-the-art accuracy across 77 IoT devices with 40 Gbps throughput, and requires only 1.3% of the processing time compared to GPU-accelerated approaches

Enabling Intrusion Detection in IPSEC Protected IPV6 Networks through Secret-Key Sharing

As the Internet Protocol version 6 (IPv6) implementation becomes more widespread, the IP Security (IPSec) features embedded into the next-generation protocol will become more accessible than ever. Though the network-layer encryption provided by IPSec is a boon to data security, its use renders standard network intrusion detection systems (NIDS) useless. The problem of performing intrusion detection on encrypted traffic has been addressed by differing means with each technique requiring one or more static secret keys to be shared with the NIDS beforehand. The problem with this approach is static keying is much less secure than dynamic key generation through the Internet Key Exchange (IKE) protocol. This research creates and evaluates a secret-key sharing framework which allows both the added security of dynamic IPSec key generation through IKE, and intrusion detection capability for a NIDS on the network. Analysis shows that network traffic related to secret-key sharing with the proposed framework can account for up to 58.6% of total traffic in the worst case scenario, though workloads which are arguably more average decrease that traffic to 10-15%. Additionally, actions associated with IKE and secret-key sharing increase CPU utilization on the NIDS up to 20.7%. Results show, at least in limited implementations, a secret-key sharing framework provides robust coverage and is a viable intrusion detection option

Accurate and Resource-Efficient Monitoring for Future Networks

Monitoring functionality is a key component of any network management system. It is essential for profiling network resource usage, detecting attacks, and capturing the performance of a multitude of services using the network. Traditional monitoring solutions operate on long timescales producing periodic reports, which are mostly used for manual and infrequent network management tasks. However, these practices have been recently questioned by the advent of Software Defined Networking (SDN). By empowering management applications with the right tools to perform automatic, frequent, and fine-grained network reconfigurations, SDN has made these applications more dependent than before on the accuracy and timeliness of monitoring reports. As a result, monitoring systems are required to collect considerable amounts of heterogeneous measurement data, process them in real-time, and expose the resulting knowledge in short timescales to network decision-making processes. Satisfying these requirements is extremely challenging given today’s larger network scales, massive and dynamic traffic volumes, and the stringent constraints on time availability and hardware resources. This PhD thesis tackles this important challenge by investigating how an accurate and resource-efficient monitoring function can be realised in the context of future, software-defined networks. Novel monitoring methodologies, designs, and frameworks are provided in this thesis, which scale with increasing network sizes and automatically adjust to changes in the operating conditions. These achieve the goal of efficient measurement collection and reporting, lightweight measurement- data processing, and timely monitoring knowledge delivery

Scalable Techniques for Anomaly Detection

Computer networks are constantly being attacked by malicious entities for various reasons. Network based attacks include but are not limited to, Distributed Denial of Service (DDoS), DNS based attacks, Cross-site Scripting (XSS) etc. Such attacks have exploited either the network protocol or the end-host software vulnerabilities for perpetration. Current network traffic analysis techniques employed for detection and/or prevention of these anomalies suffer from significant delay or have only limited scalability because of their huge resource requirements. This dissertation proposes more scalable techniques for network anomaly detection. We propose using DNS analysis for detecting a wide variety of network anomalies. The use of DNS is motivated by the fact that DNS traffic comprises only 2-3% of total network traffic reducing the burden on anomaly detection resources. Our motivation additionally follows from the observation that almost any Internet activity (legitimate or otherwise) is marked by the use of DNS. We propose several techniques for DNS traffic analysis to distinguish anomalous DNS traffic patterns which in turn identify different categories of network attacks. First, we present MiND, a system to detect misdirected DNS packets arising due to poisoned name server records or due to local infections such as caused by worms like DNSChanger. MiND validates misdirected DNS packets using an externally collected database of authoritative name servers for second or third-level domains. We deploy this tool at the edge of a university campus network for evaluation. Secondly, we focus on domain-fluxing botnet detection by exploiting the high entropy inherent in the set of domains used for locating the Command and Control (C&C) server. We apply three metrics namely the Kullback-Leibler divergence, the Jaccard Index, and the Edit distance, to different groups of domain names present in Tier-1 ISP DNS traces obtained from South Asia and South America. Our evaluation successfully detects existing domain-fluxing botnets such as Conficker and also recognizes new botnets. We extend this approach by utilizing DNS failures to improve the latency of detection. Alternatively, we propose a system which uses temporal and entropy-based correlation between successful and failed DNS queries, for fluxing botnet detection. We also present an approach which computes the reputation of domains in a bipartite graph of hosts within a network, and the domains accessed by them. The inference technique utilizes belief propagation, an approximation algorithm for marginal probability estimation. The computation of reputation scores is seeded through a small fraction of domains found in black and white lists. An application of this technique, on an HTTP-proxy dataset from a large enterprise, shows a high detection rate with low false positive rates

Modeling and Detection of Content and Packet Flow Anomalies at Enterprise Network Gateway

This dissertation investigates modeling techniques and computing algorithms for detection of anomalous contents and traffic flows of ingress Internet traffic at an enterprise network gateway. Anomalous contents refer to a large volume of ingress packets whose contents are not wanted by enterprise users, such as unsolicited electronic messages (UNE). UNE are often sent by Botnet farms for network resource exploitation, information stealing, and they incur high costs in bandwidth waste. Many products have been designed to block UNE, but most of them rely on signature database(s) for matching, and they cannot recognize unknown attacks. To address this limitation, in this dissertation I propose a Progressive E-Message Classifier (PEC) to timely classify message patterns that are commonly associated with UNE. On the basis of a scoring and aging engine, a real-time scoreboard keeps track of detected feature instances of the detection features until they are considered either as UNE or normal messages. A mathematical model has been designed to precisely depict system behaviors and then set detection parameters. The PEC performance is widely studied using different parameters based on several experiments. The objective of anomalous traffic flow detection is to detect selfish Transmission Control Protocol, TCP, flows which do not conform to one of the handful of congestion control protocols in adjusting their packet transmission rates in the face of network congestion. Given that none of the operational parameters in congestion control are carried in the transmitted packets, a gateway can only use packet arrival times to recover states of end to end congestion control rules, if any. We develop new techniques to estimate round trip time (RTT) using EWMA Lomb-Scargle periodogram, detect change of congestion windows by the CUSUM algorithm, and then finally predict detected congestion flow states using a prioritized decision chain. A high level finite state machine (FSM) takes the predictions as inputs to determine if a TCP flow follows a particular congestion control protocol. Multiple experiments show promising outcomes of classifying flows of different protocols based on the ratio of the aberrant transition count to normal transition count generated by FSM

A composable approach to design of newer techniques for large-scale denial-of-service attack attribution

Since its early days, the Internet has witnessed not only a phenomenal growth, but also a large number of security attacks, and in recent years, denial-of-service (DoS) attacks have emerged as one of the top threats. The stateless and destination-oriented Internet routing combined with the ability to harness a large number of compromised machines and the relative ease and low costs of launching such attacks has made this a hard problem to address. Additionally, the myriad requirements of scalability, incremental deployment, adequate user privacy protections, and appropriate economic incentives has further complicated the design of DDoS defense mechanisms. While the many research proposals to date have focussed differently on prevention, mitigation, or traceback of DDoS attacks, the lack of a comprehensive approach satisfying the different design criteria for successful attack attribution is indeed disturbing. Our first contribution here has been the design of a composable data model that has helped us represent the various dimensions of the attack attribution problem, particularly the performance attributes of accuracy, effectiveness, speed and overhead, as orthogonal and mutually independent design considerations. We have then designed custom optimizations along each of these dimensions, and have further integrated them into a single composite model, to provide strong performance guarantees. Thus, the proposed model has given us a single framework that can not only address the individual shortcomings of the various known attack attribution techniques, but also provide a more wholesome counter-measure against DDoS attacks. Our second contribution here has been a concrete implementation based on the proposed composable data model, having adopted a graph-theoretic approach to identify and subsequently stitch together individual edge fragments in the Internet graph to reveal the true routing path of any network data packet. The proposed approach has been analyzed through theoretical and experimental evaluation across multiple metrics, including scalability, incremental deployment, speed and efficiency of the distributed algorithm, and finally the total overhead associated with its deployment. We have thereby shown that it is realistically feasible to provide strong performance and scalability guarantees for Internet-wide attack attribution. Our third contribution here has further advanced the state of the art by directly identifying individual path fragments in the Internet graph, having adopted a distributed divide-and-conquer approach employing simple recurrence relations as individual building blocks. A detailed analysis of the proposed approach on real-life Internet topologies with respect to network storage and traffic overhead, has provided a more realistic characterization. Thus, not only does the proposed approach lend well for simplified operations at scale but can also provide robust network-wide performance and security guarantees for Internet-wide attack attribution. Our final contribution here has introduced the notion of anonymity in the overall attack attribution process to significantly broaden its scope. The highly invasive nature of wide-spread data gathering for network traceback continues to violate one of the key principles of Internet use today - the ability to stay anonymous and operate freely without retribution. In this regard, we have successfully reconciled these mutually divergent requirements to make it not only economically feasible and politically viable but also socially acceptable. This work opens up several directions for future research - analysis of existing attack attribution techniques to identify further scope for improvements, incorporation of newer attributes into the design framework of the composable data model abstraction, and finally design of newer attack attribution techniques that comprehensively integrate the various attack prevention, mitigation and traceback techniques in an efficient manner

Feedback control in intrusion detection systems

Master'sMASTER OF ENGINEERIN

Analysis and Automated Discovery of Attacks in Transport Protocols

Transport protocols like TCP and QUIC are a crucial component of today’s Internet, underlying services as diverse as email, file transfer, web browsing, video conferencing, and instant messaging as well as infrastructure protocols like BGP and secure network protocols like TLS. Transport protocols provide a variety of important guarantees like reliability, in-order delivery, and congestion control to applications. As a result, the design and implementation of transport protocols is complex, with many components, special cases, interacting features, and efficiency considerations, leading to a high probability of bugs. Unfortunately, today the testing of transport protocols is mainly a manual, ad-hoc process. This lack of systematic testing has resulted in a steady stream of attacks compromising the availability, performance, or security of transport protocols, as seen in the literature. Given the importance of these protocols, we believe that there is a need for the development of automated systems to identify complex attacks in implementations of these protocols and for a better understanding of the types of attacks that will be faced by next generation transport protocols. In this dissertation, we focus on improving this situation, and the security of transport protocols, in three ways. First, we develop a system to automatically search for attacks that target the availability or performance of protocol connections on real transport protocol implementations. Second, we implement a model-based system to search for attacks against implementations of TCP congestion control. Finally, we examine QUIC, Google’s next generation encrypted transport protocol, and identify attacks on availability and performance

Defending against low-rate TCP attack: dynamic detection and protection.

Sun Haibin.Thesis (M.Phil.)--Chinese University of Hong Kong, 2005.Includes bibliographical references (leaves 89-96).Abstracts in English and Chinese.Abstract --- p.iChinese Abstract --- p.iiiAcknowledgement --- p.ivChapter 1 --- Introduction --- p.1Chapter 2 --- Background Study and Related Work --- p.5Chapter 2.1 --- Victim Exhaustion DoS/DDoS Attacks --- p.6Chapter 2.1.1 --- Direct DoS/DDoS Attacks --- p.7Chapter 2.1.2 --- Reflector DoS/DDoS Attacks --- p.8Chapter 2.1.3 --- Spoofed Packet Filtering --- p.9Chapter 2.1.4 --- IP Traceback --- p.13Chapter 2.1.5 --- Location Hiding --- p.20Chapter 2.2 --- QoS Based DoS Attacks --- p.22Chapter 2.2.1 --- Introduction to the QoS Based DoS Attacks --- p.22Chapter 2.2.2 --- Countermeasures to the QoS Based DoS Attacks --- p.22Chapter 2.3 --- Worm based DoS Attacks --- p.24Chapter 2.3.1 --- Introduction to the Worm based DoS Attacks --- p.24Chapter 2.3.2 --- Countermeasures to the Worm Based DoS Attacks --- p.24Chapter 2.4 --- Low-rate TCP Attack and RoQ Attacks --- p.26Chapter 2.4.1 --- General Introduction of Low-rate Attack --- p.26Chapter 2.4.2 --- Introduction of RoQ Attack --- p.27Chapter 3 --- Formal Description of Low-rate TCP Attacks --- p.28Chapter 3.1 --- Mathematical Model of Low-rate TCP Attacks --- p.28Chapter 3 2 --- Other forms of Low-rate TCP Attacks --- p.31Chapter 4 --- Distributed Detection Mechanism --- p.34Chapter 4.1 --- General Consideration of Distributed Detection . --- p.34Chapter 4.2 --- Design of Low-rate Attack Detection Algorithm . --- p.36Chapter 4.3 --- Statistical Sampling of Incoming Traffic --- p.37Chapter 4.4 --- Noise Filtering --- p.38Chapter 4.5 --- Feature Extraction --- p.39Chapter 4.6 --- Pattern Matching via the Dynamic Time Warping (DTW) Method --- p.41Chapter 4.7 --- Robustness and Accuracy of DTW --- p.45Chapter 4.7.1 --- DTW values for low-rate attack: --- p.46Chapter 4.7.2 --- DTW values for legitimate traffic (Gaussian): --- p.47Chapter 4.7.3 --- DTW values for legitimate traffic (Self-similar): --- p.48Chapter 5 --- Low-Rate Attack Defense Mechanism --- p.52Chapter 5.1 --- Design of Defense Mechanism --- p.52Chapter 5.2 --- Analysis of Deficit Round Robin Algorithm --- p.54Chapter 6 --- Fluid Model of TCP Flows --- p.56Chapter 6.1 --- Fluid Math. Model of TCP under DRR --- p.56Chapter 6.1.1 --- Model of TCP on a Droptail Router --- p.56Chapter 6.1.2 --- Model of TCP on a DRR Router --- p.60Chapter 6.2 --- Simulation of TCP Fluid Model --- p.62Chapter 6.2.1 --- Simulation of Attack with Single TCP Flow --- p.62Chapter 6.2.2 --- Simulation of Attack with Multiple TCP flows --- p.64Chapter 7 --- Experiments --- p.69Chapter 7.1 --- Experiment 1 (Single TCP flow vs. single source attack) --- p.69Chapter 7.2 --- Experiment 2 (Multiple TCP flows vs. single source attack) --- p.72Chapter 7.3 --- Experiment 3 (Multiple TCP flows vs. synchro- nized distributed low-rate attack) --- p.74Chapter 7.4 --- Experiment 4 (Network model of low-rate attack vs. Multiple TCP flows) --- p.77Chapter 8 --- Conclusion --- p.83Chapter A --- Lemmas and Theorem Derivation --- p.85Bibliography --- p.8

Techniques for Processing TCP/IP Flow Content in Network Switches at Gigabit Line Rates

The growth of the Internet has enabled it to become a critical component used by businesses, governments and individuals. While most of the traffic on the Internet is legitimate, a proportion of the traffic includes worms, computer viruses, network intrusions, computer espionage, security breaches and illegal behavior. This rogue traffic causes computer and network outages, reduces network throughput, and costs governments and companies billions of dollars each year. This dissertation investigates the problems associated with TCP stream processing in high-speed networks. It describes an architecture that simplifies the processing of TCP data streams in these environments and presents a hardware circuit capable of TCP stream processing on multi-gigabit networks for millions of simultaneous network connections. Live Internet traffic is analyzed using this new TCP processing circuit