Towards Large-Scale, Heterogeneous Anomaly Detection Systems in Industrial Networks: A Survey of Current Trends

Industrial Networks (INs) are widespread environments where heterogeneous devices collaborate to control and monitor physical processes. Some of the controlled processes belong to Critical Infrastructures (CIs), and, as such, IN protection is an active research field. Among different types of security solutions, IN Anomaly Detection Systems (ADSs) have received wide attention from the scientific community.While INs have grown in size and in complexity, requiring the development of novel, Big Data solutions for data processing, IN ADSs have not evolved at the same pace. In parallel, the development of BigData frameworks such asHadoop or Spark has led the way for applying Big Data Analytics to the field of cyber-security,mainly focusing on the Information Technology (IT) domain. However, due to the particularities of INs, it is not feasible to directly apply IT security mechanisms in INs, as IN ADSs face unique characteristics. In this work we introduce three main contributions. First, we survey the area of Big Data ADSs that could be applicable to INs and compare the surveyed works. Second, we develop a novel taxonomy to classify existing INbased ADSs. And, finally, we present a discussion of open problems in the field of Big Data ADSs for INs that can lead to further development

Command & Control: Understanding, Denying and Detecting - A review of malware C2 techniques, detection and defences

In this survey, we first briefly review the current state of cyber attacks, highlighting significant recent changes in how and why such attacks are performed. We then investigate the mechanics of malware command and control (C2) establishment: we provide a comprehensive review of the techniques used by attackers to set up such a channel and to hide its presence from the attacked parties and the security tools they use. We then switch to the defensive side of the problem, and review approaches that have been proposed for the detection and disruption of C2 channels. We also map such techniques to widely-adopted security controls, emphasizing gaps or limitations (and success stories) in current best practices.Comment: Work commissioned by CPNI, available at c2report.org. 38 pages. Listing abstract compressed from version appearing in repor

On the Scalable Generation of Cyber Threat Intelligence from Passive DNS Streams

Domain Name System (DNS) has become an important element of recent cybercrime infrastructures. Indeed, DNS protocol is being used, for instance, to operate infected machines and transport malicious payloads. In this context, it is of paramount importance to analyze passive DNS streams in order to generate timely and relevant cyber threat intelligence that can be used to detect, prevent and attribute cyber attacks. In this thesis, we explore the analysis of the aforementioned streams in order to detect DNS anomalies that correspond to cyber incidents. By DNS anomaly, we mean any deviation from what is expected in terms of regular DNS activities (queries/responses). The identification of these anomalies leads to precious intelligence that could pinpoint domains that are involved in malicious activities (e.g., spamming, botnets, phishing, DDoS, etc.). We propose, design and implement a system that analyzes, in near-real-time, passive DNS streams and generates cyber threat intelligence in terms of: suspicious domains, DNS record abuse and passive DNS anomalies. We correlate the generated intelligence with other sources of intelligence such as our malware database. We dedicate a special care to the scalability of the proposed system. In addition to picking appropriate data structures and database technologies, we proceed with the distribution of the analysis over a cluster of computers using the so-called map/reduce paradigm with the Apache Spark framework. Our experiments show that our system is efficient and scalable while generating important, relevant and timely cyber threat intelligence

A Survey on Enterprise Network Security: Asset Behavioral Monitoring and Distributed Attack Detection

Enterprise networks that host valuable assets and services are popular and frequent targets of distributed network attacks. In order to cope with the ever-increasing threats, industrial and research communities develop systems and methods to monitor the behaviors of their assets and protect them from critical attacks. In this paper, we systematically survey related research articles and industrial systems to highlight the current status of this arms race in enterprise network security. First, we discuss the taxonomy of distributed network attacks on enterprise assets, including distributed denial-of-service (DDoS) and reconnaissance attacks. Second, we review existing methods in monitoring and classifying network behavior of enterprise hosts to verify their benign activities and isolate potential anomalies. Third, state-of-the-art detection methods for distributed network attacks sourced from external attackers are elaborated, highlighting their merits and bottlenecks. Fourth, as programmable networks and machine learning (ML) techniques are increasingly becoming adopted by the community, their current applications in network security are discussed. Finally, we highlight several research gaps on enterprise network security to inspire future research.Comment: Journal paper submitted to Elseive

Scalable Techniques for Anomaly Detection

Computer networks are constantly being attacked by malicious entities for various reasons. Network based attacks include but are not limited to, Distributed Denial of Service (DDoS), DNS based attacks, Cross-site Scripting (XSS) etc. Such attacks have exploited either the network protocol or the end-host software vulnerabilities for perpetration. Current network traffic analysis techniques employed for detection and/or prevention of these anomalies suffer from significant delay or have only limited scalability because of their huge resource requirements. This dissertation proposes more scalable techniques for network anomaly detection. We propose using DNS analysis for detecting a wide variety of network anomalies. The use of DNS is motivated by the fact that DNS traffic comprises only 2-3% of total network traffic reducing the burden on anomaly detection resources. Our motivation additionally follows from the observation that almost any Internet activity (legitimate or otherwise) is marked by the use of DNS. We propose several techniques for DNS traffic analysis to distinguish anomalous DNS traffic patterns which in turn identify different categories of network attacks. First, we present MiND, a system to detect misdirected DNS packets arising due to poisoned name server records or due to local infections such as caused by worms like DNSChanger. MiND validates misdirected DNS packets using an externally collected database of authoritative name servers for second or third-level domains. We deploy this tool at the edge of a university campus network for evaluation. Secondly, we focus on domain-fluxing botnet detection by exploiting the high entropy inherent in the set of domains used for locating the Command and Control (C&C) server. We apply three metrics namely the Kullback-Leibler divergence, the Jaccard Index, and the Edit distance, to different groups of domain names present in Tier-1 ISP DNS traces obtained from South Asia and South America. Our evaluation successfully detects existing domain-fluxing botnets such as Conficker and also recognizes new botnets. We extend this approach by utilizing DNS failures to improve the latency of detection. Alternatively, we propose a system which uses temporal and entropy-based correlation between successful and failed DNS queries, for fluxing botnet detection. We also present an approach which computes the reputation of domains in a bipartite graph of hosts within a network, and the domains accessed by them. The inference technique utilizes belief propagation, an approximation algorithm for marginal probability estimation. The computation of reputation scores is seeded through a small fraction of domains found in black and white lists. An application of this technique, on an HTTP-proxy dataset from a large enterprise, shows a high detection rate with low false positive rates

Intelligent conditional collaborative private data sharing

With the advent of distributed systems, secure and privacy-preserving data sharing between different entities (individuals or organizations) becomes a challenging issue. There are several real-world scenarios in which different entities are willing to share their private data only under certain circumstances, such as sharing the system logs when there is indications of cyber attack in order to provide cyber threat intelligence. Therefore, over the past few years, several researchers proposed solutions for collaborative data sharing, mostly based on existing cryptographic algorithms. However, the existing approaches are not appropriate for conditional data sharing, i.e., sharing the data if and only if a pre-defined condition is satisfied due to the occurrence of an event. Moreover, in case the existing solutions are used in conditional data sharing scenarios, the shared secret will be revealed to all parties and re-keying process is necessary. In this work, in order to address the aforementioned challenges, we propose, a “conditional collaborative private data sharing” protocol based on Identity-Based Encryption and Threshold Secret Sharing schemes. In our proposed approach, the condition based on which the encrypted data will be revealed to the collaborating parties (or a central entity) could be of two types: (i) threshold, or (ii) pre-defined policy. Supported by thorough analytical and experimental analysis, we show the effectiveness and performance of our proposal