Improving intrusion detection systems using data mining techniques

Recent surveys and studies have shown that cyber-attacks have caused a lot of damage to organisations, governments, and individuals around the world. Although developments are constantly occurring in the computer security field, cyber-attacks still cause damage as they are developed and evolved by hackers. This research looked at some industrial challenges in the intrusion detection area. The research identified two main challenges; the first one is that signature-based intrusion detection systems such as SNORT lack the capability of detecting attacks with new signatures without human intervention. The other challenge is related to multi-stage attack detection, it has been found that signature-based is not efficient in this area. The novelty in this research is presented through developing methodologies tackling the mentioned challenges. The first challenge was handled by developing a multi-layer classification methodology. The first layer is based on decision tree, while the second layer is a hybrid module that uses two data mining techniques; neural network, and fuzzy logic. The second layer will try to detect new attacks in case the first one fails to detect. This system detects attacks with new signatures, and then updates the SNORT signature holder automatically, without any human intervention. The obtained results have shown that a high detection rate has been obtained with attacks having new signatures. However, it has been found that the false positive rate needs to be lowered. The second challenge was approached by evaluating IP information using fuzzy logic. This approach looks at the identity of participants in the traffic, rather than the sequence and contents of the traffic. The results have shown that this approach can help in predicting attacks at very early stages in some scenarios. However, it has been found that combining this approach with a different approach that looks at the sequence and contents of the traffic, such as event- correlation, will achieve a better performance than each approach individually

Polygraph: Automatically generating signatures for polymorphic worms

It is widely believed that content-signature-based intrusion detection systems (IDSes) are easily evaded by polymorphic worms, which vary their payload on every infection attempt. In this paper, we present Polygraph, a signature generation system that successfully produces signatures that match polymorphic worms. Polygraph generates signatures that consist of multiple disjoint content sub-strings. In doing so, Polygraph leverages our insight that for a real-world exploit to function properly, multiple invariant substrings must often be present in all variants of a payload; these substrings typically correspond to protocol framing, return addresses, and in some cases, poorly obfuscated code. We contribute a definition of the polymorphic signature generation problem; propose classes of signature suited for matching polymorphic worm payloads; and present algorithms for automatic generation of signatures in these classes. Our evaluation of these algorithms on a range of polymorphic worms demonstrates that Polygraph produces signatures for polymorphic worms that exhibit low false negatives and false positives. © 2005 IEEE

BEHAVIORAL CHARACTERIZATION OF ATTACKS ON THE REMOTE DESKTOP PROTOCOL

The Remote Desktop Protocol (RDP) is popular for enabling remote access and administration of Windows systems; however, attackers can take advantage of RDP to cause harm to critical systems using it. Detection and classification of RDP attacks is a challenge because most RDP traffic is encrypted, and it is not always clear which connections to a system are malicious after manual decryption of RDP traffic. In this research, we used open-source tools to generate and analyze RDP attack data using a power-grid honeypot under our control. We developed methods for detecting and characterizing RDP attacks through malicious signatures, Windows event log entries, and network traffic metadata. Testing and evaluation of our characterization methods on actual attack data collected by four instances of our honeypot showed that we could effectively delineate benign and malicious RDP traffic and classify the severity of RDP attacks on unprotected or misconfigured Windows systems. The classification of attack patterns and severity levels can inform defenders of adversarial behavior in RDP attacks. Our results can also help protect national critical infrastructure, including Department of Defense systems.DOE, Washington DC 20805Civilian, SFSApproved for public release. Distribution is unlimited

Botnet lab creation with open source tools and usefulness of such a tool for researchers

Botnets are large scale networks, which can span across the internet and comprise of computers, which have been infected by malicious software and are centrally controlled from a remote location. Botnets pose a great security risk and their size has been rising drastically over the past few years. The use of botnets by the underground community as a medium for online crime, bundled with their use for profit has shined the spotlight on them. Numerous researchers have proposed and designed infrastructures and frameworks that identify newly formed botnets and their traffic patterns. In this research, the design of a unified modular open source laboratory is proposed, with the use of virtual machines and open source tools, which can be used in analyzing and dissecting newly found bots in the wild. Furthermore, the usefulness and flexibility of the open source laboratory is evaluated by infecting my test machines with the Zeus Bot