A behavioural study in runtime analysis environments and drive-by download attacks

In the information age, the growth in availability of both technology and exploit kits have continuously contributed in a large volume of websites being compromised or set up with malicious intent. The issue of drive-by-download attacks formulate a high percentage (77%) of the known attacks against client systems. These attacks originate from malicious web-servers or compromised web-servers and attack client systems by pushing malware upon interaction. Within the detection and intelligence gathering area of research, high-interaction honeypot approaches have been a longstanding and well-established technology. These are however not without challenges: analysing the entirety of the world wide web using these approaches is unviable due to time and resource intensiveness. Furthermore, the volume of data that is generated as a result of a run-time analysis of the interaction between website and an analysis environment is huge, varied and not well understood. The volume of malicious servers in addition to the large datasets created as a result of run-time analysis are contributing factors in the difficulty of analysing and verifying actual malicious behaviour. The work in this thesis attempts to overcome the difficulties in the analysis process of log files to optimise malicious and anomaly behaviour detection. The main contribution of this work is focused on reducing the volume of data generated from run-time analysis to reduce the impact of noise within behavioural log file datasets. This thesis proposes an alternate approach that uses an expert lead approach to filtering benign behaviour from potentially malicious and unknown behaviour. Expert lead filtering is designed in a risk-averse method that takes into account known benign and expected behaviours before filtering the log file. Moreover, the approach relies upon behavioural investigation as well as potential for 5 system compromisation before filtering out behaviour within dynamic analysis log files. Consequently, this results in a significantly lower volume of data that can be analysed in greater detail. The proposed filtering approach has been implemented and tested in real-world context using a prudent experimental framework. An average of 96.96% reduction in log file size has been achieved which is transferable to behaviour analysis environments. The other contributions of this work include the understanding of observable operating system interactions. Within the study of behaviour analysis environments, it was concluded that run-time analysis environments are sensitive to application and operating system versions. Understanding key changes in operating systems behaviours within Windows is an unexplored area of research yet Windows is currently one of the most popular client operating system. As part of understanding system behaviours for the creation of behavioural filters, this study undertakes a number of experiments to identify the key behaviour differences between operating systems. The results show that there are significant changes in core processes and interactions which can be taken into account in the development of filters for updated systems. Finally, from the analysis of 110,000 potentially malicious websites, typical attacks are explored. These attacks actively exploited the honeypot and offer knowledge on a section of the active web-based attacks faced in the world wide web. Trends and attack vectors are identified and evaluated