Survey of Intrusion Detection Research

The literature holds a great deal of research in the intrusion detection area. Much of this describes the design and implementation of specific intrusion detection systems. While the main focus has been the study of different detection algorithms and methods, there are a number of other issues that are of equal importance to make these systems function well in practice. I believe that the reason that the commercial market does not use many of the ideas described is that there are still too many unresolved issues. This survey focuses on presenting the different issues that must be addressed to build fully functional and practically usable intrusion detection systems (IDSs). It points out the state of the art in each area and suggests important open research issues

A GENERIC ARCHITECTURE FOR INSIDER MISUSE MONITORING IN IT SYSTEMS

Intrusion Detection Systems (IDS) have been widely deployed within many organisations' IT nenvorks to delect network penetration attacks by outsiders and privilege escalation attacks by insiders. However, traditional IDS are ineffective for detecting o f abuse o f legitimate privileges by authorised users within the organisation i.e. the detection of misfeasance. In essence insider IT abuse does not violate system level controls, yet violates acceptable usage policy, business controls, or code of conduct defined by the organisation. However, the acceptable usage policy can vary from one organisation to another, and the acceptability o f user activities can also change depending upon the user(s), application, machine, data, and other contextual conditions associated with the entities involved. The fact that the perpetrators are authorised users and that the insider misuse activities do not violate system level controls makes detection of insider abuse more complicated than detection o f attacks by outsiders. The overall aim o f the research is to determine novel methods by which monitoring and detection may be improved to enable successful detection of insider IT abuse. The discussion begins with a comprehensive investigation o f insider IT misuse, encompassing the breadth and scale of the problem. Consideration is then given to the sufficiency of existing safeguards, with the conclusion that they provide an inadequate basis for detecting many o f the problems. This finding is used as the justification for considering research into alternative approaches. The realisation of the research objective includes the development of a taxonomy for identification o f various levels within the system from which the relevant data associated with each type of misuse can be collected, and formulation of a checklist for identification of applications that requires misfeasor monitoring. Based upon this foundation a novel architecture for monitoring o f insider IT misuse, has been designed. The design offers new analysis procedures to be added, while providing methods to include relevant contextual parameters from dispersed systems for analysis and reference. The proposed system differs from existing IDS in the way that it focuses on detecting contextual misuse of authorised privileges and legitimate operations, rather than detecting exploitation o f network protocols and system level \ailnerabilities. The main concepts of the new architecture were validated through a proof-of-concept prototype system. A number o f case scenarios were used to demonstrate the validity of analysis procedures developed and how the contextual data from dispersed databases can be used for analysis of various types of insider activities. This helped prove that the existing detection technologies can be adopted for detection o f insider IT misuse, and that the research has thus provided valuable contribution to the domain

AI Solutions for MDS: Artificial Intelligence Techniques for Misuse Detection and Localisation in Telecommunication Environments

This report considers the application of Articial Intelligence (AI) techniques to the problem of misuse detection and misuse localisation within telecommunications environments. A broad survey of techniques is provided, that covers inter alia rule based systems, model-based systems, case based reasoning, pattern matching, clustering and feature extraction, articial neural networks, genetic algorithms, arti cial immune systems, agent based systems, data mining and a variety of hybrid approaches. The report then considers the central issue of event correlation, that is at the heart of many misuse detection and localisation systems. The notion of being able to infer misuse by the correlation of individual temporally distributed events within a multiple data stream environment is explored, and a range of techniques, covering model based approaches, `programmed' AI and machine learning paradigms. It is found that, in general, correlation is best achieved via rule based approaches, but that these suffer from a number of drawbacks, such as the difculty of developing and maintaining an appropriate knowledge base, and the lack of ability to generalise from known misuses to new unseen misuses. Two distinct approaches are evident. One attempts to encode knowledge of known misuses, typically within rules, and use this to screen events. This approach cannot generally detect misuses for which it has not been programmed, i.e. it is prone to issuing false negatives. The other attempts to `learn' the features of event patterns that constitute normal behaviour, and, by observing patterns that do not match expected behaviour, detect when a misuse has occurred. This approach is prone to issuing false positives, i.e. inferring misuse from innocent patterns of behaviour that the system was not trained to recognise. Contemporary approaches are seen to favour hybridisation, often combining detection or localisation mechanisms for both abnormal and normal behaviour, the former to capture known cases of misuse, the latter to capture unknown cases. In some systems, these mechanisms even work together to update each other to increase detection rates and lower false positive rates. It is concluded that hybridisation offers the most promising future direction, but that a rule or state based component is likely to remain, being the most natural approach to the correlation of complex events. The challenge, then, is to mitigate the weaknesses of canonical programmed systems such that learning, generalisation and adaptation are more readily facilitated

Analysis of digital evidence in identity theft investigations

Identity Theft could be currently considered as a significant problem in the modern internet driven era. This type of computer crime can be achieved in a number of different ways; various statistical figures suggest it is on the increase. It intimidates individual privacy and self assurance, while efforts for increased security and protection measures appear inadequate to prevent it. A forensic analysis of the digital evidence should be able to provide precise findings after the investigation of Identity Theft incidents. At present, the investigation of Internet based Identity Theft is performed on an ad hoc and unstructured basis, in relation to the digital evidence. This research work aims to construct a formalised and structured approach to digital Identity Theft investigations that would improve the current computer forensic investigative practice. The research hypothesis is to create an analytical framework to facilitate the investigation of Internet Identity Theft cases and the processing of the related digital evidence. This research work makes two key contributions to the subject: a) proposing the approach of examining different computer crimes using a process specifically based on their nature and b) to differentiate the examination procedure between the victim’s and the fraudster’s side, depending on the ownership of the digital media. The background research on the existing investigation methods supports the need of moving towards an individual framework that supports Identity Theft investigations. The presented investigation framework is designed based on the structure of the existing computer forensic frameworks. It is a flexible, conceptual tool that will assist the investigator’s work and analyse incidents related to this type of crime. The research outcome has been presented in detail, with supporting relevant material for the investigator. The intention is to offer a coherent tool that could be used by computer forensics investigators. Therefore, the research outcome will not only be evaluated from a laboratory experiment, but also strengthened and improved based on an evaluation feedback by experts from law enforcement. While personal identities are increasingly being stored and shared on digital media, the threat of personal and private information that is used fraudulently cannot be eliminated. However, when such incidents are precisely examined, then the nature of the problem can be more clearly understood

An Insider Misuse Threat Detection and Prediction Language

Numerous studies indicate that amongst the various types of security threats, the problem of insider misuse of IT systems can have serious consequences for the health of computing infrastructures. Although incidents of external origin are also dangerous, the insider IT misuse problem is difficult to address for a number of reasons. A fundamental reason that makes the problem mitigation difficult relates to the level of trust legitimate users possess inside the organization. The trust factor makes it difficult to detect threats originating from the actions and credentials of individual users. An equally important difficulty in the process of mitigating insider IT threats is based on the variability of the problem. The nature of Insider IT misuse varies amongst organizations. Hence, the problem of expressing what constitutes a threat, as well as the process of detecting and predicting it are non trivial tasks that add up to the multi- factorial nature of insider IT misuse. This thesis is concerned with the process of systematizing the specification of insider threats, focusing on their system-level detection and prediction. The design of suitable user audit mechanisms and semantics form a Domain Specific Language to detect and predict insider misuse incidents. As a result, the thesis proposes in detail ways to construct standardized descriptions (signatures) of insider threat incidents, as means of aiding researchers and IT system experts mitigate the problem of insider IT misuse. The produced audit engine (LUARM – Logging User Actions in Relational Mode) and the Insider Threat Prediction and Specification Language (ITPSL) are two utilities that can be added to the IT insider misuse mitigation arsenal. LUARM is a novel audit engine designed specifically to address the needs of monitoring insider actions. These needs cannot be met by traditional open source audit utilities. ITPSL is an XML based markup that can standardize the description of incidents and threats and thus make use of the LUARM audit data. Its novelty lies on the fact that it can be used to detect as well as predict instances of threats, a task that has not been achieved to this date by a domain specific language to address threats. The research project evaluated the produced language using a cyber-misuse experiment approach derived from real world misuse incident data. The results of the experiment showed that the ITPSL and its associated audit engine LUARM provide a good foundation for insider threat specification and prediction. Some language deficiencies relate to the fact that the insider threat specification process requires a good knowledge of the software applications used in a computer system. As the language is easily expandable, future developments to improve the language towards this direction are suggested

Augmenting security event information with contextual data to improve the detection capabilities of a SIEM

The increasing number of cyber security breaches have revealed a need for proper cyber security measures. The emergence of the internet and the increase in overall connectivity means that data is more easily accessible and available. Using the available data in a security context may provide a system with an external contextual insight such as environmental awareness or current affair awareness. A security information and event management (SIEM) system is a security system that correlates security event information from surrounding systems and decides whether the surrounding environment (possibly an enterprise's network) is vulnerable or even under attack by a malicious person whether they be internal (authorised) or external (unauthorised). In this thesis, the aim is to provide such a system with con- text by adding non-security related information from surrounding available sources known as context information feeds. Contextual information feeds are added to the SIEM and tested using randomised events. There are various context information types used in this thesis, namely: social media, meteorological, calendar information and terror threat level. The SIEM is tested with each contextual data feed active and the results are recorded. The testing shows that the addition of contextual data feeds actively affects the sensitivity of OSSIM and hence results in higher alarms raised during elevated context triggered states. The system showed a greater and deeper visibility of its surrounding environment and hence an improved detection capability

Modulating application behaviour for closely coupled intrusion detection

Includes bibliographical references.This thesis presents a security measure that is closely coupled to applications. This distinguishes it from conventional security measures which tend to operate at the infrastructure level (network, operating system or virtual machine). Such lower level mechanisms exhibit a number of limitations, amongst others they are poorly suited to the monitoring of applications which operate on encrypted data or the enforcement of security policies involving abstractions introduced by applications. In order to address these problems, the thesis proposes externalising the security related analysis functions performed by applications. These otherwise remain hidden in applications and so are likely to be underdeveloped, inflexible or insular. It is argued that these deficiencies have resulted in an over-reliance on infrastructure security components

Detecting worm mutations using machine learning

Worms are malicious programs that spread over the Internet without human intervention. Since worms generally spread faster than humans can respond, the only viable defence is to automate their detection. Network intrusion detection systems typically detect worms by examining packet or flow logs for known signatures. Not only does this approach mean that new worms cannot be detected until the corresponding signatures are created, but that mutations of known worms will remain undetected because each mutation will usually have a different signature. The intuitive and seemingly most effective solution is to write more generic signatures, but this has been found to increase false alarm rates and is thus impractical. This dissertation investigates the feasibility of using machine learning to automatically detect mutations of known worms. First, it investigates whether Support Vector Machines can detect mutations of known worms. Support Vector Machines have been shown to be well suited to pattern recognition tasks such as text categorisation and hand-written digit recognition. Since detecting worms is effectively a pattern recognition problem, this work investigates how well Support Vector Machines perform at this task. The second part of this dissertation compares Support Vector Machines to other machine learning techniques in detecting worm mutations. Gaussian Processes, unlike Support Vector Machines, automatically return confidence values as part of their result. Since confidence values can be used to reduce false alarm rates, this dissertation determines how Gaussian Process compare to Support Vector Machines in terms of detection accuracy. For further comparison, this work also compares Support Vector Machines to K-nearest neighbours, known for its simplicity and solid results in other domains. The third part of this dissertation investigates the automatic generation of training data. Classifier accuracy depends on good quality training data -- the wider the training data spectrum, the higher the classifier's accuracy. This dissertation describes the design and implementation of a worm mutation generator whose output is fed to the machine learning techniques as training data. This dissertation then evaluates whether the training data can be used to train classifiers of sufficiently high quality to detect worm mutations. The findings of this work demonstrate that Support Vector Machines can be used to detect worm mutations, and that the optimal configuration for detection of worm mutations is to use a linear kernel with unnormalised bi-gram frequency counts. Moreover, the results show that Gaussian Processes and Support Vector Machines exhibit similar accuracy on average in detecting worm mutations, while K-nearest neighbours consistently produces lower quality predictions. The generated worm mutations are shown to be of sufficiently high quality to serve as training data. Combined, the results demonstrate that machine learning is capable of accurately detecting mutations of known worms

Non-business use of the World Wide Web : A study of selected Western Australian organisations

Employees undertake a wide range of activities when they use the World Wide Web in the work place. Some of these activities may leave the modem Internet connected organisation vulnerable to undue or unknown risk, potential productivity losses and expense us a result of misuse or abuse or the Internet provision. Much of the existing literature on this subject points to a purported epidemic of misuse in the workplace. If this practice is so prevalent and widespread, what can modem Internet connected organisations do to identify the abuse and reduce the risks and losses that these abuses represent? To what extent is the World Wide Web used by employees for non-business related activities in organisations and can filtering or organisational policies impact on this activity? This research specifically examines contextually, the level of misuse with respect to the use of the World Wide Web in three selected Western Australian organisations using multiple interpretive case study as the vehicle for the study. The research is significant internationally to all organisations that use Internet in their everyday work. The research has discovered anomalous behaviour on the part of non-business users who have employed a variety of techniques and tactics to mask their activities. Also, organisational management in the cases examined had demonstrated shortfalls in their perception of misuse within their organisations and, the implementation of effective policy

Assessing and augmenting SCADA cyber security: a survey of techniques

SCADA systems monitor and control critical infrastructures of national importance such as power generation and distribution, water supply, transportation networks, and manufacturing facilities. The pervasiveness, miniaturisations and declining costs of internet connectivity have transformed these systems from strictly isolated to highly interconnected networks. The connectivity provides immense benefits such as reliability, scalability and remote connectivity, but at the same time exposes an otherwise isolated and secure system, to global cyber security threats. This inevitable transformation to highly connected systems thus necessitates effective security safeguards to be in place as any compromise or downtime of SCADA systems can have severe economic, safety and security ramifications. One way to ensure vital asset protection is to adopt a viewpoint similar to an attacker to determine weaknesses and loopholes in defences. Such mind sets help to identify and fix potential breaches before their exploitation. This paper surveys tools and techniques to uncover SCADA system vulnerabilities. A comprehensive review of the selected approaches is provided along with their applicability