Can we beat legitimate cyber behavior mimicking attacks from botnets?

Abstract—Botnets are the engine for malicious activities in cyber space. In order to sustain their botnets and disguise their illegal actions, botnet owners are exhausting their strength to mimic legitimate cyber behavior to fly under the radar, e.g. flash crowd mimicking attacks on popular websites. It is an open and challenging problem: can we beat mimicking attacks or not? We use web browsing on popular websites as an example to explore the issue. In our previous work, we discovered that it is almost impossible to detect mimicking attacks from statistics if the number of active bots of a botnet is sufficient (no less than the number of active legitimate users). In this paper, we pointed out that it is usually hard for botnet owners to have sufficient number of active bots in practice. Therefore, we can discriminate mimicking attacks when the sufficient number condition is not met. We prove our claim theoretically and confirm it with simulations. Our findings can also be applied to a large number of other detection related cases. Index Terms—mimicking attack; flash crowd attack; botnet; detection. I

An intelligent, distributed and collaborative DDoS defense system

The Distributed Denial-of-Service (DDoS) attack is known as one of the most destructive attacks on the Internet. With the advent of new computing paradigms, such as Cloud and Mobile computing, and the emergence of pervasive technology, such as the Internet of Things, on one hand, these revolutionized technologies enable the availability of services and applications to everyone. On the other hand, these techniques also benefit attackers to exploit the vulnerabilities and deploy attacks in more efficient ways. Latest network security reports have shown that distributed Denial of Service (DDoS) attacks have been growing dramatically in volume, frequency, sophistication and impact, making it one of the most challenging threats in the Internet. An unfortunate state of affairs is that the remediation strategies have fallen behind attackers. The severe impact caused by recent DDoS attacks strongly indicates the need for an effective DDoS defense system. We study the current existing solution space, and summarize three fundamental requirements for an effective DDoS defense system: 1) an accurate detection with minimal false alarms; 2) an effective inline inspection and instant mitigation, and 3) a dynamic, distributed and collaborative defense infrastructure. This thesis aims at providing such a defense system that fulfills all the requirements. In this thesis, we explore and address the problem from three directions: 1) we strive to understand the existing detection strategies and provide a survey of an empirical analysis of machine learning based detection techniques; 2) we develop a novel hybrid detection model which ensembles a deep learning model for a practical flow by flow detection and a classic machine learning model that is aware of the network status, and 3) we present the design and implementation of an intelligent, distributed and collaborative DDoS defense system that effectively mitigate the impact of DDoS attacks. The performance evaluation results show that our proposed defense system is capable of effectively mitigating DDoS attacks impacts and maintaining a limited disturbing for legitimate services

Using Visual Analytics to Discover Bot Traffic

With the advance of technology, the Internet has become a medium tool used for many malicious activities. The presence of bot traffic has increased greatly that causes significant problems for businesses and organisations, such as spam bots, scraper bots, distributed denial of service bots and adaptive bots that aim to exploit the vulnerabilities of a website. Discriminating bot traffic against legitimate flash crowds remains an open challenge to date.In order to address the above issues and enhance security awareness, this thesis proposes an interactive visual analytics system for discovering bot traffic. The system provides an interactive visualisation, with details on demand capabilities, which enables knowledge discovery from very large datasets. It enables an analyst to understand comprehensive details without being constrained by large datasets. The system has a dashboard view to represent legitimate and bot traffic by adopting Quadtree data structure and Voronoi diagrams. The main contribution of this thesis is a novel visual analytics system that is capable of discovering bot traffic.This research conducted a literature review in order to gain systematic understanding of the research area. Furthermore, the research was conducted by utilising experiment and simulation approaches. The experiment was conducted by capturing website traffic, identifying browser fingerprints, simulating bot attacks and analysing mouse dynamics, such as movements and events, of participants. Data were captured as the participants performed a list of tasks, such as responding to the banner. The data collection is transparent to the participants and only requires JavaScript to be activated on the client side. This study involved 10 participants who are familiar with the Internet. To analyse the data, Weka 3.6.10 was used to perform classification based on a training dataset. The test dataset of all participants was evaluated using a built-in decision tree algorithm. The results of classifying the test dataset were promising, and the model was able to identify ten participants and six simulated bot attacks with an accuracy of 86.67%. Finally, the visual analytics design was formulated in order to assist an analyst to discover bot presence

Denial-of-service attack modelling and detection for HTTP/2 services

Businesses and society alike have been heavily dependent on Internet-based services, albeit with experiences of constant and annoying disruptions caused by the adversary class. A malicious attack that can prevent establishment of Internet connections to web servers, initiated from legitimate client machines, is termed as a Denial of Service (DoS) attack; volume and intensity of which is rapidly growing thanks to the readily available attack tools and the ever-increasing network bandwidths. A majority of contemporary web servers are built on the HTTP/1.1 communication protocol. As a consequence, all literature found on DoS attack modelling and appertaining detection techniques, addresses only HTTP/1.x network traffic. This thesis presents a model of DoS attack traffic against servers employing the new communication protocol, namely HTTP/2. The HTTP/2 protocol significantly differs from its predecessor and introduces new messaging formats and data exchange mechanisms. This creates an urgent need to understand how malicious attacks including Denial of Service, can be launched against HTTP/2 services. Moreover, the ability of attackers to vary the network traffic models to stealthy affects web services, thereby requires extensive research and modelling. This research work not only provides a novel model for DoS attacks against HTTP/2 services, but also provides a model of stealthy variants of such attacks, that can disrupt routine web services. Specifically, HTTP/2 traffic patterns that consume computing resources of a server, such as CPU utilisation and memory consumption, were thoroughly explored and examined. The study presents four HTTP/2 attack models. The first being a flooding-based attack model, the second being a distributed model, the third and fourth are variant DoS attack models. The attack traffic analysis conducted in this study employed four machine learning techniques, namely Naïve Bayes, Decision Tree, JRip and Support Vector Machines. The HTTP/2 normal traffic model portrays online activities of human users. The model thus formulated was employed to also generate flash-crowd traffic, i.e. a large volume of normal traffic that incapacitates a web server, similar in fashion to a DoS attack, albeit with non-malicious intent. Flash-crowd traffic generated based on the defined model was used to populate the dataset of legitimate network traffic, to fuzz the machine learning-based attack detection process. The two variants of DoS attack traffic differed in terms of the traffic intensities and the inter-packet arrival delays introduced to better analyse the type and quality of DoS attacks that can be launched against HTTP/2 services. A detailed analysis of HTTP/2 features is also presented to rank relevant network traffic features for all four traffic models presented. These features were ranked based on legitimate as well as attack traffic observations conducted in this study. The study shows that machine learning-based analysis yields better classification performance, i.e. lower percentage of incorrectly classified instances, when the proposed HTTP/2 features are employed compared to when HTTP/1.1 features alone are used. The study shows how HTTP/2 DoS attack can be modelled, and how future work can extend the proposed model to create variant attack traffic models that can bypass intrusion-detection systems. Likewise, as the Internet traffic and the heterogeneity of Internet-connected devices are projected to increase significantly, legitimate traffic can yield varying traffic patterns, demanding further analysis. The significance of having current legitimate traffic datasets, together with the scope to extend the DoS attack models presented herewith, suggest that research in the DoS attack analysis and detection area will benefit from the work presented in this thesis