Using Botnet Technologies to Counteract Network Traffic Analysis

Botnets have been problematic for over a decade. They are used to launch malicious activities including DDoS (Distributed-Denial-of-Service), spamming, identity theft, unauthorized bitcoin mining and malware distribution. A recent nation-wide DDoS attacks caused by the Mirai botnet on 10/21/2016 involving 10s of millions of IP addresses took down Twitter, Spotify, Reddit, The New York Times, Pinterest, PayPal and other major websites. In response to take-down campaigns by security personnel, botmasters have developed technologies to evade detection. The most widely used evasion technique is DNS fast-flux, where the botmaster frequently changes the mapping between domain names and IP addresses of the C&C server so that it will be too late or too costly to trace the C&C server locations. Domain names generated with Domain Generation Algorithms (DGAs) are used as the \u27rendezvous\u27 points between botmasters and bots. This work focuses on how to apply botnet technologies (fast-flux and DGA) to counteract network traffic analysis, therefore protecting user privacy. A better understanding of botnet technologies also helps us be pro-active in defending against botnets. First, we proposed two new DGAs using hidden Markov models (HMMs) and Probabilistic Context-Free Grammars (PCFGs) which can evade current detection methods and systems. Also, we developed two HMM-based DGA detection methods that can detect the botnet DGA-generated domain names with/without training sets. This helps security personnel understand the botnet phenomenon and develop pro-active tools to detect botnets. Second, we developed a distributed proxy system using fast-flux to evade national censorship and surveillance. The goal is to help journalists, human right advocates and NGOs in West Africa to have a secure and free Internet. Then we developed a covert data transport protocol to transform arbitrary message into real DNS traffic. We encode the message into benign-looking domain names generated by an HMM, which represents the statistical features of legitimate domain names. This can be used to evade Deep Packet Inspection (DPI) and protect user privacy in a two-way communication. Both applications serve as examples of applying botnet technologies to legitimate use. Finally, we proposed a new protocol obfuscation technique by transforming arbitrary network protocol into another (Network Time Protocol and a video game protocol of Minecraft as examples) in terms of packet syntax and side-channel features (inter-packet delay and packet size). This research uses botnet technologies to help normal users have secure and private communications over the Internet. From our botnet research, we conclude that network traffic is a malleable and artificial construct. Although existing patterns are easy to detect and characterize, they are also subject to modification and mimicry. This means that we can construct transducers to make any communication pattern look like any other communication pattern. This is neither bad nor good for security. It is a fact that we need to accept and use as best we can

Command & Control: Understanding, Denying and Detecting - A review of malware C2 techniques, detection and defences

In this survey, we first briefly review the current state of cyber attacks, highlighting significant recent changes in how and why such attacks are performed. We then investigate the mechanics of malware command and control (C2) establishment: we provide a comprehensive review of the techniques used by attackers to set up such a channel and to hide its presence from the attacked parties and the security tools they use. We then switch to the defensive side of the problem, and review approaches that have been proposed for the detection and disruption of C2 channels. We also map such techniques to widely-adopted security controls, emphasizing gaps or limitations (and success stories) in current best practices.Comment: Work commissioned by CPNI, available at c2report.org. 38 pages. Listing abstract compressed from version appearing in repor

Detecting DNS Threats: A Deep Learning Model to Rule Them All

Domain Name Service is a central part of Internet regular operation. Such importance has made it a common target of different malicious behaviors such as the application of Domain Generation Algorithms (DGA) for command and control a group of infected computers or Tunneling techniques for bypassing system administrator restrictions. A common detection approach is based on training different models detecting DGA and Tunneling capable of performing a lexicographic discrimination of the domain names. However, since both DGA and Tunneling showed domain names with observable lexicographical differences with normal domains, it is reasonable to apply the same detection approach to both threats. In the present work, we propose a multi-class convolutional network (MC-CNN) capable of detecting both DNS threats. The resulting MC-CNN is able to detect correctly 99% of normal domains, 97% of DGA and 92% of Tunneling, with a False Positive Rate of 2.8%, 0.7% and 0.0015% respectively and the advantage of having 44% fewer trainable parameters than similar models applied to DNS threats detection.Sociedad Argentina de Informática e Investigación Operativ

Detecting DNS Threats: A Deep Learning Model to Rule Them All

Domain Name Service is a central part of Internet regular operation. Such importance has made it a common target of different malicious behaviors such as the application of Domain Generation Algorithms (DGA) for command and control a group of infected computers or Tunneling techniques for bypassing system administrator restrictions. A common detection approach is based on training different models detecting DGA and Tunneling capable of performing a lexicographic discrimination of the domain names. However, since both DGA and Tunneling showed domain names with observable lexicographical differences with normal domains, it is reasonable to apply the same detection approach to both threats. In the present work, we propose a multi-class convolutional network (MC-CNN) capable of detecting both DNS threats. The resulting MC-CNN is able to detect correctly 99% of normal domains, 97% of DGA and 92% of Tunneling, with a False Positive Rate of 2.8%, 0.7% and 0.0015% respectively and the advantage of having 44% fewer trainable parameters than similar models applied to DNS threats detection.Sociedad Argentina de Informática e Investigación Operativ

Encrypted and Covert DNS Queries for Botnets: Challenges and Countermeasures

There is a continuous increase in the sophistication that modern malware exercise in order to bypass the deployed security mechanisms. A typical approach to evade the identification and potential take down of a botnet command and control server is domain fluxing through the use of Domain Generation Algorithms (DGAs). These algorithms produce a vast amount of domain names that the infected device tries to communicate with to find the C&C server, yet only a small fragment of them is actually registered. This allows the botmaster to pivot the control and make the work of seizing the botnet control rather difficult. Current state of the art and practice considers that the DNS queries performed by a compromised device are transparent to the network administrator and therefore can be monitored, analysed, and blocked. In this work, we showcase that the latter is a strong assumption as malware could efficiently hide its DNS queries using covert and/or encrypted channels bypassing the detection mechanisms. To this end, we discuss possible mitigation measures based on traffic analysis to address the new challenges that arise from this approach

Artificial intelligence in the cyber domain: Offense and defense

Artificial intelligence techniques have grown rapidly in recent years, and their applications in practice can be seen in many fields, ranging from facial recognition to image analysis. In the cybersecurity domain, AI-based techniques can provide better cyber defense tools and help adversaries improve methods of attack. However, malicious actors are aware of the new prospects too and will probably attempt to use them for nefarious purposes. This survey paper aims at providing an overview of how artificial intelligence can be used in the context of cybersecurity in both offense and defense.Web of Science123art. no. 41

Monitoring security of enterprise hosts via DNS data analysis

Enterprise Networks are growing in scale and complexity, with heterogeneous connected assets needing to be secured in different ways. Nevertheless, virtually all connected assets use the Domain Name System (DNS) for address resolution. Thus DNS has become a convenient vehicle for attackers to covertly perform Command and Control (C&C) communication, data theft, and service disruption across a wide range of assets. Enterprise security appliances that monitor network traffic typically allow all DNS traffic through as it is vital for accessing any web service; they may at best match against a database of known malicious patterns, and are therefore ineffective against zero-day attacks. This thesis focuses on three high-impact cyber-attacks that leverage DNS, specifically data exfiltration, malware C&C communication, and service disruption. Using big data (over 10B packets) of DNS network traffic collected from a University campus and a Government research organization over six months, we illustrate the anatomy of these attacks, train machines for automatically detecting such attacks, and evaluate their efficacy in the field. The contributions of this thesis are three-fold: Our first contribution tackles data exfiltration using DNS. We analyze outgoing DNS queries to identify many stateless attributes such as the number of characters, the number of labels, and the entropy of the domain name to distinguish malicious data exfiltration queries from legitimate ones. We train our machines using ground-truth obtained from a public list of top 10K legitimate domains and empirically validate and tune our models to achieve over 98% accuracy in correctly distinguish legitimate DNS queries from malicious ones, the latter coming from known malware domains as well as synthetically generated using popular DNS exfiltration tools. Our second contribution tackles malware C&C communication using DNS. We analyze DNS outgoing queries to identify more than twenty families of DGA (Domain Generation Algorithm)-enabled malware when communicating with their C&C servers. We identify attributes of network traffic that commences following the resolution of a DGA-based DNS query. We train three protocol-specific one-class classifier models, for HTTP, HTTPS and UDP flows, using public packet traces of known malware. We develop a monitoring system that uses reactive rules to automatically and selectively mirror TCP/UDP flows (between internal hosts and malware servers) pertinent to DGA queries for diagnosis by the trained models. We deploy our system in the field and evaluate its performance to show that it flags more than 2000 internal assets as potentially infected, generating more than a million suspicious flows, of which more than 97% are verified to be malicious by an off-the-shelf intrusion detection system. Our third contribution studies the use of DNS for service disruption. We analyze incoming DNS messages, with a specific focus on non-existent (NXD) DNS responses, to distinguish benign from malicious NXDs. We highlight two attack scenarios based on their requested domain names. Using NXD behavioral attributes of internal hosts, we develop multi-staged iForest classification models to detect internal hosts launching service disruption attacks. We show how our models can detect infected hosts that generate high-volume and low-volume distributed NXD-based attacks on public resolvers and/or authoritative name servers with an accuracy of over 99% in correctly classifying legitimate hosts. Our work shines a light on a critical vector in enterprise security and equips the enterprise network operator with the means to detect and block sophisticated attackers who use DNS as a vehicle for malware C&C communication, data exfiltration, and service disruption

Detecting DNS Threats: A Deep Learning Model to Rule Them All

Domain Name Service is a central part of Internet regular operation. Such importance has made it a common target of different malicious behaviors such as the application of Domain Generation Algorithms (DGA) for command and control a group of infected computers or Tunneling techniques for bypassing system administrator restrictions. A common detection approach is based on training different models detecting DGA and Tunneling capable of performing a lexicographic discrimination of the domain names. However, since both DGA and Tunneling showed domain names with observable lexicographical differences with normal domains, it is reasonable to apply the same detection approach to both threats. In the present work, we propose a multi-class convolutional network (MC-CNN) capable of detecting both DNS threats. The resulting MC-CNN is able to detect correctly 99% of normal domains, 97% of DGA and 92% of Tunneling, with a False Positive Rate of 2.8%, 0.7% and 0.0015% respectively and the advantage of having 44% fewer trainable parameters than similar models applied to DNS threats detection.Sociedad Argentina de Informática e Investigación Operativ