Multi-Stage Detection Technique for DNS-Based Botnets

Domain Name System (DNS) is one of the most widely used protocols in the Internet. The main purpose of the DNS protocol is mapping user-friendly domain names to IP addresses. Unfortunately, many cyber criminals deploy the DNS protocol for malicious purposes, such as botnet communications. In this type of attack, the botmasters tunnel communications between the Command and Control (C&C) servers and the bot-infected machines within DNS request and response. Designing an effective approach for botnet detection has been done previously based on specific botnet types Since botnet communications are characterized by different features, botmasters may evade detection methods by modifying some of these features. This research aims to design and implement a multi-staged detection approach for Domain Generation Algorithm (DGA), Fast Flux Service Network, and Domain Flux-based botnets, as well as encrypted DNS tunneled-based botnets using the BRO Network Security Monitor. This approach is able to detect DNS-based botnet communications by relying on analyzing different techniques used for finding the C&C server, as well as encrypting the malicious traffic

Detection of Early-Stage Enterprise Infection by Mining Large-Scale Log Data

Recent years have seen the rise of more sophisticated attacks including advanced persistent threats (APTs) which pose severe risks to organizations and governments by targeting confidential proprietary information. Additionally, new malware strains are appearing at a higher rate than ever before. Since many of these malware are designed to evade existing security products, traditional defenses deployed by most enterprises today, e.g., anti-virus, firewalls, intrusion detection systems, often fail at detecting infections at an early stage. We address the problem of detecting early-stage infection in an enterprise setting by proposing a new framework based on belief propagation inspired from graph theory. Belief propagation can be used either with "seeds" of compromised hosts or malicious domains (provided by the enterprise security operation center -- SOC) or without any seeds. In the latter case we develop a detector of C&C communication particularly tailored to enterprises which can detect a stealthy compromise of only a single host communicating with the C&C server. We demonstrate that our techniques perform well on detecting enterprise infections. We achieve high accuracy with low false detection and false negative rates on two months of anonymized DNS logs released by Los Alamos National Lab (LANL), which include APT infection attacks simulated by LANL domain experts. We also apply our algorithms to 38TB of real-world web proxy logs collected at the border of a large enterprise. Through careful manual investigation in collaboration with the enterprise SOC, we show that our techniques identified hundreds of malicious domains overlooked by state-of-the-art security products

Framework for botnet emulation and analysis

Criminals use the anonymity and pervasiveness of the Internet to commit fraud, extortion, and theft. Botnets are used as the primary tool for this criminal activity. Botnets allow criminals to accumulate and covertly control multiple Internet-connected computers. They use this network of controlled computers to flood networks with traffic from multiple sources, send spam, spread infection, spy on users, commit click fraud, run adware, and host phishing sites. This presents serious privacy risks and financial burdens to businesses and individuals. Furthermore, all indicators show that the problem is worsening because the research and development cycle of the criminal industry is faster than that of security research. To enable researchers to measure botnet connection models and counter-measures, a flexible, rapidly augmentable framework for creating test botnets is provided. This botnet framework, written in the Ruby language, enables researchers to run a botnet on a closed network and to rapidly implement new communication, spreading, control, and attack mechanisms for study. This is a significant improvement over augmenting C++ code-bases for the most popular botnets, Agobot and SDBot. Rubot allows researchers to implement new threats and their corresponding defenses before the criminal industry can. The Rubot experiment framework includes models for some of the latest trends in botnet operation such as peer-to-peer based control, fast-flux DNS, and periodic updates. Our approach implements the key network features from existing botnets and provides the required infrastructure to run the botnet in a closed environment.Ph.D.Committee Chair: Copeland, John; Committee Member: Durgin, Gregory; Committee Member: Goodman, Seymour; Committee Member: Owen, Henry; Committee Member: Riley, Georg

Advances in modern botnet understanding and the accurate enumeration of infected hosts

Botnets remain a potent threat due to evolving modern architectures, inadequate remediation methods, and inaccurate measurement techniques. In response, this re- search exposes the architectures and operations of two advanced botnets, techniques to enumerate infected hosts, and pursues the scientific refinement of infected-host enu- meration data by recognizing network structures which distort measurement. This effort is motivated by the desire to reveal botnet behavior and trends for future mit- igation, methods to discover infected hosts for remediation in real time and threat assessment, and the need to reveal the inaccuracy in population size estimation when only counting IP addresses. Following an explanation of theoretical enumeration techniques, the architectures, deployment methodologies, and malicious output for the Storm and Waledac botnets are presented. Several tools developed to enumerate these botnets are then assessed in terms of performance and yield. Finally, this study documents methods that were developed to discover the boundaries and impact of NAT and DHCP blocks in network populations along with a footprint measurement based on relative entropy which better describes how uniformly infections communi- cate through their IP addresses. Population data from the Waledac botnet was used to evaluate these techniqu

On the Scalable Generation of Cyber Threat Intelligence from Passive DNS Streams

Domain Name System (DNS) has become an important element of recent cybercrime infrastructures. Indeed, DNS protocol is being used, for instance, to operate infected machines and transport malicious payloads. In this context, it is of paramount importance to analyze passive DNS streams in order to generate timely and relevant cyber threat intelligence that can be used to detect, prevent and attribute cyber attacks. In this thesis, we explore the analysis of the aforementioned streams in order to detect DNS anomalies that correspond to cyber incidents. By DNS anomaly, we mean any deviation from what is expected in terms of regular DNS activities (queries/responses). The identification of these anomalies leads to precious intelligence that could pinpoint domains that are involved in malicious activities (e.g., spamming, botnets, phishing, DDoS, etc.). We propose, design and implement a system that analyzes, in near-real-time, passive DNS streams and generates cyber threat intelligence in terms of: suspicious domains, DNS record abuse and passive DNS anomalies. We correlate the generated intelligence with other sources of intelligence such as our malware database. We dedicate a special care to the scalability of the proposed system. In addition to picking appropriate data structures and database technologies, we proceed with the distribution of the analysis over a cluster of computers using the so-called map/reduce paradigm with the Apache Spark framework. Our experiments show that our system is efficient and scalable while generating important, relevant and timely cyber threat intelligence

An analysis of the use of DNS for malicious payload distribution

The Domain Name System (DNS) protocol is a fundamental part of Internet activities that can be abused by cybercriminals to conduct malicious activities. Previous research has shown that cybercriminals use different methods, including the DNS protocol, to distribute malicious content, remain hidden and avoid detection from various technologies that are put in place to detect anomalies. This allows botnets and certain malware families to establish covert communication channels that can be used to send or receive data and also distribute malicious payloads using the DNS queries and responses. Cybercriminals use the DNS to breach highly protected networks, distribute malicious content, and exfiltrate sensitive information without being detected by security controls put in place by embedding certain strings in DNS packets. This research undertaking broadens this research field and fills in the existing research gap by extending the analysis of DNS being used as a payload distribution channel to detection of domains that are used to distribute different malicious payloads. This research undertaking analysed the use of the DNS in detecting domains and channels that are used for distributing malicious payloads. Passive DNS data which replicate DNS queries on name servers to detect anomalies in DNS queries was evaluated and analysed in order to detect malicious payloads. The research characterises the malicious payload distribution channels by analysing passive DNS traffic and modelling the DNS query and response patterns. The research found that it is possible to detect malicious payload distribution channels through the analysis of DNS TXT resource records

Network-Based Detection and Prevention System against DNS-Based Attacks

Individuals and organizations rely on the Internet as an essential environment for personal or business transactions. However, individuals and organizations have been primary targets for attacks that steal sensitive data. Adversaries can use different approaches to hide their activities inside the compromised network and communicate covertly between the malicious servers and the victims. The domain name system (DNS) protocol is one of these approaches that adversaries use to transfer stolen data outside the organization\u27s network using various forms of DNS tunneling attacks. The main reason for targeting the DNS protocol is because DNS is available in almost every network, ignored, and rarely monitored. In this work, the primary aim is to design a reliable and robust network-based solution as a detection system against DNS-based attacks using various techniques, including visualization, machine learning techniques, and statistical analysis. The network-based solution acts as a DNS proxy server that provides DNS services as well as detection and prevention against DNS-based attacks, which are either embedded in malware or used as stand-alone attacking tools. The detection system works in two modes: real-time and offline modes. The real-time mode relies on the developed Payload Analysis (PA) module. In contrast, the offline mode operates based on two of the contributed modules in this dissertation, including the visualization and Traffic Analysis (TA) modules. We conducted various experiments in order to test and evaluate the detection system against simulated real-world attacks. Overall, the detection system achieved high accuracy of 99.8% with no false-negative rate. To validate the method, we compared the developed detection system against the open-source detection system, Snort intrusion detection system (IDS). We evaluated the two detection systems using a confusion matrix, including the recall, false-negatives rate, accuracy, and others. The detection system detects all case scenarios of the attacks while Snort missed 50% of the performed attacks. Based on the results, we can conclude that the detection system is significant and original improvement of the present methods used for detecting and preventing DNS-based attacks

Using Botnet Technologies to Counteract Network Traffic Analysis

Botnets have been problematic for over a decade. They are used to launch malicious activities including DDoS (Distributed-Denial-of-Service), spamming, identity theft, unauthorized bitcoin mining and malware distribution. A recent nation-wide DDoS attacks caused by the Mirai botnet on 10/21/2016 involving 10s of millions of IP addresses took down Twitter, Spotify, Reddit, The New York Times, Pinterest, PayPal and other major websites. In response to take-down campaigns by security personnel, botmasters have developed technologies to evade detection. The most widely used evasion technique is DNS fast-flux, where the botmaster frequently changes the mapping between domain names and IP addresses of the C&C server so that it will be too late or too costly to trace the C&C server locations. Domain names generated with Domain Generation Algorithms (DGAs) are used as the \u27rendezvous\u27 points between botmasters and bots. This work focuses on how to apply botnet technologies (fast-flux and DGA) to counteract network traffic analysis, therefore protecting user privacy. A better understanding of botnet technologies also helps us be pro-active in defending against botnets. First, we proposed two new DGAs using hidden Markov models (HMMs) and Probabilistic Context-Free Grammars (PCFGs) which can evade current detection methods and systems. Also, we developed two HMM-based DGA detection methods that can detect the botnet DGA-generated domain names with/without training sets. This helps security personnel understand the botnet phenomenon and develop pro-active tools to detect botnets. Second, we developed a distributed proxy system using fast-flux to evade national censorship and surveillance. The goal is to help journalists, human right advocates and NGOs in West Africa to have a secure and free Internet. Then we developed a covert data transport protocol to transform arbitrary message into real DNS traffic. We encode the message into benign-looking domain names generated by an HMM, which represents the statistical features of legitimate domain names. This can be used to evade Deep Packet Inspection (DPI) and protect user privacy in a two-way communication. Both applications serve as examples of applying botnet technologies to legitimate use. Finally, we proposed a new protocol obfuscation technique by transforming arbitrary network protocol into another (Network Time Protocol and a video game protocol of Minecraft as examples) in terms of packet syntax and side-channel features (inter-packet delay and packet size). This research uses botnet technologies to help normal users have secure and private communications over the Internet. From our botnet research, we conclude that network traffic is a malleable and artificial construct. Although existing patterns are easy to detect and characterize, they are also subject to modification and mimicry. This means that we can construct transducers to make any communication pattern look like any other communication pattern. This is neither bad nor good for security. It is a fact that we need to accept and use as best we can

Malicious Payload Distribution Channels in Domain Name System

Botmasters are known to use different protocols to hide their activities under the radar. Throughout the past years, several protocols have been abused and recently Domain Name System (DNS) also became a target of such malicious activities. In this dissertation, we analyze the use of DNS as a malicious payload distribution channel. To the best of our knowledge, this is the first comprehensive analysis of these payload distribution channels via DNS. We present a system to characterize such channels in the passive DNS (pDNS) traffic by modelling DNS query and response patterns. Then, we analyze the Resource Record (RR) activities of these channels to build their DNS zone profiles. Finally, we detect and assign levels of intensity for payload distribution channels by using a fuzzy logic theory. Our work is based on an extensive analysis of malware datasets for one year, and a near real-time feed of pDNS traffic. The experimental results reveal few long-running hidden domains used by Morto worm to distribute malicious payloads. We also found that some of these payloads are in cleartext, without any encoding or encryption. Our experiments on pDNS traffic indicate that our system can detect these channels regardless of the payload format. Passive DNS is a useful data source for DNS based research, and it requires to be stored in a database for historical data analysis, such as the work we present in this dissertation. Once this database is established, it can be used for any sort of threat analysis that requires DNS oriented intelligence. Our aim is to create a scalable pDNS database, that contains potentially valuable security intelligence data. We present our pDNS database by discussing the database design, implementation challenges, and the evaluation of the system