A Characterization of Cybersecurity Posture from Network Telescope Data

Data-driven understanding of cybersecurity posture is an important problem that has not been adequately explored. In this paper, we analyze some real data collected by CAIDA's network telescope during the month of March 2013. We propose to formalize the concept of cybersecurity posture from the perspectives of three kinds of time series: the number of victims (i.e., telescope IP addresses that are attacked), the number of attackers that are observed by the telescope, and the number of attacks that are observed by the telescope. Characterizing cybersecurity posture therefore becomes investigating the phenomena and statistical properties exhibited by these time series, and explaining their cybersecurity meanings. For example, we propose the concept of {\em sweep-time}, and show that sweep-time should be modeled by stochastic process, rather than random variable. We report that the number of attackers (and attacks) from a certain country dominates the total number of attackers (and attacks) that are observed by the telescope. We also show that substantially smaller network telescopes might not be as useful as a large telescope

Visualizing big network traffic data using frequent pattern mining and hypergraphs

Visualizing communication logs, like NetFlow records, is extremely useful for numerous tasks that need to analyze network traffic traces, like network planning, performance monitoring, and troubleshooting. Communication logs, however, can be massive, which necessitates designing effective visualization techniques for large data sets. To address this problem, we introduce a novel network traffic visualization scheme based on the key ideas of (1) exploiting frequent itemset mining (FIM) to visualize a succinct set of interesting traffic patterns extracted from large traces of communication logs; and (2) visualizing extracted patterns as hypergraphs that clearly display multi-attribute associations. We demonstrate case studies that support the utility of our visualization scheme and show that it enables the visualization of substantially larger data sets than existing network traffic visualization schemes based on parallel-coordinate plots or graphs. For example, we show that our scheme can easily visualize the patterns of more than 41 million NetFlow records. Previous research has explored using parallel-coordinate plots for visualizing network traffic flows. However, such plots do not scale to data sets with thousands of even millions of flows

Distributed Data Streaming Algorithms for Network Anomaly Detection

Network attacks and anomalies such as DDoS attacks, service outages, email spamming are happening everyday, causing various problems for users such as financial loss, inconvenience due to service unavailability, personal information leakage and so on. Different methods have been studied and developed to tackle these network attacks, and among them data streaming algorithms are quite powerful, useful and flexible schemes that have many applications in network attack detection and identification. Data streaming algorithms usually use limited space to store aggregated information and report certain properties of the traffic in short and constant time. There are several challenges for designing data streaming algorithms. Firstly, network traffic is usually distributed and monitored at different locations, and it is often desirable to aggregate the distributed monitoring information together to detect attacks which might be low-profile at a single location; thus data streaming algorithms have to support data merging without loss of information. Secondly, network traffic is usually in high-speed and large-volume; data streaming algorithms have to process data fast and smart to save space and time. Thirdly, sometimes only detection is not useful enough and identification of targets make more sense, in which case data streaming algorithms have to be concise and reversible. In this dissertation, we study three different types of data streaming algorithms: hot item identification, distinct element counting and superspreader identification. We propose new algorithms to solve these problems and evaluate them with both theoretical analysis and experiments to show their effectiveness and improvements upon previous methods

Resilience Strategies for Network Challenge Detection, Identification and Remediation

The enormous growth of the Internet and its use in everyday life make it an attractive target for malicious users. As the network becomes more complex and sophisticated it becomes more vulnerable to attack. There is a pressing need for the future internet to be resilient, manageable and secure. Our research is on distributed challenge detection and is part of the EU Resumenet Project (Resilience and Survivability for Future Networking: Framework, Mechanisms and Experimental Evaluation). It aims to make networks more resilient to a wide range of challenges including malicious attacks, misconfiguration, faults, and operational overloads. Resilience means the ability of the network to provide an acceptable level of service in the face of significant challenges; it is a superset of commonly used definitions for survivability, dependability, and fault tolerance. Our proposed resilience strategy could detect a challenge situation by identifying an occurrence and impact in real time, then initiating appropriate remedial action. Action is autonomously taken to continue operations as much as possible and to mitigate the damage, and allowing an acceptable level of service to be maintained. The contribution of our work is the ability to mitigate a challenge as early as possible and rapidly detect its root cause. Also our proposed multi-stage policy based challenge detection system identifies both the existing and unforeseen challenges. This has been studied and demonstrated with an unknown worm attack. Our multi stage approach reduces the computation complexity compared to the traditional single stage, where one particular managed object is responsible for all the functions. The approach we propose in this thesis has the flexibility, scalability, adaptability, reproducibility and extensibility needed to assist in the identification and remediation of many future network challenges

Evaluating Sociotechnical Factors Associated With Telecom Service Provisioning: A Case Study

Provisioning Internet services remains an area of concern for Internet service providers. Despite investments to improve resources and technology, the understanding of sociotechnical factors that influence the service-provisioning life cycle remains limited. The purpose of this case study was to evaluate the influence of sociotechnical factors associated with telecom service provisioning and to explore the critical success and failure factors, specifically in the telecommunication industry of Kuwait. Guided by sociotechnical systems theory, this qualitative exploratory case study approach examined a purposeful sample of 19 participants comprising of managers, engineers, and technicians who had the knowledge and experience of the service-provisioning life cycle. Semistructured interviews, project logs, and a self-created follow-up questionnaire were the primary sources of data. Thematic analysis techniques assisted in coding the data and developing themes, which resulted in a set of critical success and failure factors that influence the service-provisioning life cycle. Cross-functional communication, risk management practices, infrastructure availability, and employee skill development were among the emergent factors that influenced the service implementation. Internet service providers may use the results from this study to improve the service-provisioning life cycle. Successful implementations will promote an environment of positive social change that will increase employee motivation, productivity, and employee morale

A Brave New World: Studies on the Deployment and Security of the Emerging IPv6 Internet.

Recent IPv4 address exhaustion events are ushering in a new era of rapid transition to the next generation Internet protocol---IPv6. Via Internet-scale experiments and data analysis, this dissertation characterizes the adoption and security of the emerging IPv6 network. The work includes three studies, each the largest of its kind, examining various facets of the new network protocol's deployment, routing maturity, and security. The first study provides an analysis of ten years of IPv6 deployment data, including quantifying twelve metrics across ten global-scale datasets, and affording a holistic understanding of the state and recent progress of the IPv6 transition. Based on cross-dataset analysis of relative global adoption rates and across features of the protocol, we find evidence of a marked shift in the pace and nature of adoption in recent years and observe that higher-level metrics of adoption lag lower-level metrics. Next, a network telescope study covering the IPv6 address space of the majority of allocated networks provides insight into the early state of IPv6 routing. Our analyses suggest that routing of average IPv6 prefixes is less stable than that of IPv4. This instability is responsible for the majority of the captured misdirected IPv6 traffic. Observed dark (unallocated destination) IPv6 traffic shows substantial differences from the unwanted traffic seen in IPv4---in both character and scale. Finally, a third study examines the state of IPv6 network security policy. We tested a sample of 25 thousand routers and 520 thousand servers against sets of TCP and UDP ports commonly targeted by attackers. We found systemic discrepancies between intended security policy---as codified in IPv4---and deployed IPv6 policy. Such lapses in ensuring that the IPv6 network is properly managed and secured are leaving thousands of important devices more vulnerable to attack than before IPv6 was enabled. Taken together, findings from our three studies suggest that IPv6 has reached a level and pace of adoption, and shows patterns of use, that indicates serious production employment of the protocol on a broad scale. However, weaker IPv6 routing and security are evident, and these are leaving early dual-stack networks less robust than the IPv4 networks they augment.PhDComputer Science and EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/120689/1/jczyz_1.pd

Data-Driven Approaches for Detecting Malware-Infected IoT Devices and Characterizing Their Unsolicited Behaviors by Leveraging Passive Internet Measurements

Despite the benefits of Internet of Things (IoT) devices, the insecurity of IoT and their deployment nature have turned them into attractive targets for adversaries, which contributed to the rise of IoT-tailored malware as a major threat to the Internet ecosystem. In this thesis, we address the threats associated with the emerging IoT malware, which utilize exploited devices to perform large-scale cyber attacks (e.g., DDoS). To mitigate such threat, there is a need to possess an Internet perspective of the deployed IoT devices while building a better understanding about the behavioral characteristic of malware-infected devices, which is challenging due to the lack of empirical data and knowledge about the deployed IoT devices and their behavioral characteristics. To address these challenges, in this thesis, we leverage passive Internet measurements and IoT device information to detect exploited IoT devices and investigate their generated traffic at the network telescope (darknet). We aim at proposing data-driven approaches for effective and near real-time IoT threat detection and characterization. Additionally, we leverage a specialized IoT Honeypot to analyze a large corpus of real IoT malware binary executable. We aim at building a better understanding about the current state of IoT malware while addressing the problems of IoT malware classification and family attribution. To this end, we perform the following to achieve our objectives: First, we address the lack of empirical data and knowledge about IoT devices and their activities. To this end, we leverage an online IoT search engine (e.g., Shodan.io) to obtain publicly available device information in the realms of consumer and cyber-physical system (CPS), while utilizing passive network measurements collected at a large-scale network telescope (CAIDA), to infer compromised devices and their unsolicited activities. Indeed, we were among the first to report experimental results on detecting compromised IoT devices and their behavioral characteristics in the wild, while demonstrating their active involvement in large-scale malware-generated malicious activities such as Internet scanning. Additionally, we leverage the IoT-generated backscatter traffic towards the network telescope to shed light on IoT devices that were victims of intensive Denial of Service (DoS) attacks. Second, given the highly orchestrated nature of IoT-driven cyber-attacks, we focus on the analysis of IoT-generated scanning activities to detect and characterize scanning campaigns generated by IoT botnets. To this end, we aggregate IoT-generated traffic and performing association rules mining to infer campaigns through common scanning objectives represented by targeted destination ports. Further, we leverage behavioural characteristics and aggregated flow features to correlate IoT devices using DBSCAN clustering algorithm. Indeed, our findings shed light on compromised IoT devices, which tend to operate within well coordinated IoT botnets. Third, considering the huge number of IoT devices and the magnitude of their malicious scanning traffic, we focus on addressing the operational challenges to automate large-scale campaign detection and analysis while generating threat intelligence in a timely manner. To this end, we leverage big data analytic frameworks such as Apache Spark to develop a scalable system for automated detection of infected IoT devices and characterization of their scanning activities using our proposed approach. Our evaluation results with over 4TB of IoT traffic demonstrated the effectiveness of the system to infer scanning campaigns generated by IoT botnets. Moreover, we demonstrate the feasibility of the implemented system/framework as a platform for implementing further supporting applications, which leverage passive Internet measurement for characterizing IoT traffic and generating IoT-related threat intelligence. Fourth, we take first steps towards mitigating threats associated with the rise of IoT malware by creating a better understanding about the characteristics and inter-relations of IoT malware. To this end, we analyze about 70,000 IoT malware binaries obtained by a specialized IoT honeypot in the past two years. We investigate the distribution of IoT malware across known families, while exploring their detection timeline and persistent. Moreover, while we shed light on the effectiveness of IoT honeypots in detecting new/unknown malware samples, we utilize static and dynamic malware analysis techniques to uncover adversarial infrastructure and investigate functional similarities. Indeed, our findings enable unknown malware labeling/attribution while identifying new IoT malware variants. Additionally, we collect malware-generated scanning traffic (whenever available) to explore behavioral characteristics and associated threats/vulnerabilities. We conclude this thesis by discussing research gaps that pave the way for future work