AI Solutions for MDS: Artificial Intelligence Techniques for Misuse Detection and Localisation in Telecommunication Environments

This report considers the application of Articial Intelligence (AI) techniques to the problem of misuse detection and misuse localisation within telecommunications environments. A broad survey of techniques is provided, that covers inter alia rule based systems, model-based systems, case based reasoning, pattern matching, clustering and feature extraction, articial neural networks, genetic algorithms, arti cial immune systems, agent based systems, data mining and a variety of hybrid approaches. The report then considers the central issue of event correlation, that is at the heart of many misuse detection and localisation systems. The notion of being able to infer misuse by the correlation of individual temporally distributed events within a multiple data stream environment is explored, and a range of techniques, covering model based approaches, `programmed' AI and machine learning paradigms. It is found that, in general, correlation is best achieved via rule based approaches, but that these suffer from a number of drawbacks, such as the difculty of developing and maintaining an appropriate knowledge base, and the lack of ability to generalise from known misuses to new unseen misuses. Two distinct approaches are evident. One attempts to encode knowledge of known misuses, typically within rules, and use this to screen events. This approach cannot generally detect misuses for which it has not been programmed, i.e. it is prone to issuing false negatives. The other attempts to `learn' the features of event patterns that constitute normal behaviour, and, by observing patterns that do not match expected behaviour, detect when a misuse has occurred. This approach is prone to issuing false positives, i.e. inferring misuse from innocent patterns of behaviour that the system was not trained to recognise. Contemporary approaches are seen to favour hybridisation, often combining detection or localisation mechanisms for both abnormal and normal behaviour, the former to capture known cases of misuse, the latter to capture unknown cases. In some systems, these mechanisms even work together to update each other to increase detection rates and lower false positive rates. It is concluded that hybridisation offers the most promising future direction, but that a rule or state based component is likely to remain, being the most natural approach to the correlation of complex events. The challenge, then, is to mitigate the weaknesses of canonical programmed systems such that learning, generalisation and adaptation are more readily facilitated

Multi-step scenario matching based on unification

This paper presents an approach to multi-step scenario specification and matching, which aims to address some of the issues and problems inherent in to scenario specification and event correlation found in most previous work. Our approach builds upon the unification algorithm which we have adapted to provide a seamless, integrated mechanism and framework to handle event matching, filtering, and correlation. Scenario specifications using our framework need to contain only a definition of the misuse activity to be matched. This characteristic differentiates our work from most of the previous work which generally requires scenario specifications also to include additional information regarding how to detect the misuse activity. In this paper we present a prototype implementation which demonstrates the effectiveness of the unification-based approach and our scenario specification framework. Also, we evaluate the practical usability of the approac

Min–Max Hyperellipsoidal Clustering for Anomaly Detection in Network Security

A novel hyperellipsoidal clustering technique is presented for an intrusion-detection system in network security. Hyperellipsoidal clusters toward maximum intracluster similarity and minimum intercluster similarity are generated from training data sets. The novelty of the technique lies in the fact that the parameters needed to construct higher order data models in general multivariate Gaussian functions are incrementally derived from the data sets using accretive processes. The technique is implemented in a feedforward neural network that uses a Gaussian radial basis function as the model generator. An evaluation based on the inclusiveness and exclusiveness of samples with respect to specific criteria is applied to accretively learn the output clusters of the neural network. One significant advantage of this is its ability to detect individual anomaly types that are hard to detect with other anomaly-detection schemes. Applying this technique, several feature subsets of the tcptrace network-connection records that give above 95% detection at false-positive rates below 5% were identified

Predicting Network Attacks Using Ontology-Driven Inference

Graph knowledge models and ontologies are very powerful modeling and re asoning tools. We propose an effective approach to model network attacks and attack prediction which plays important roles in security management. The goals of this study are: First we model network attacks, their prerequisites and consequences using knowledge representation methods in order to provide description logic reasoning and inference over attack domain concepts. And secondly, we propose an ontology-based system which predicts potential attacks using inference and observing information which provided by sensory inputs. We generate our ontology and evaluate corresponding methods using CAPEC, CWE, and CVE hierarchical datasets. Results from experiments show significant capability improvements comparing to traditional hierarchical and relational models. Proposed method also reduces false alarms and improves intrusion detection effectiveness.Comment: 9 page

Системы обнаружения вторжения

Оглянуто існуючі системи виявлення вторгнень, надана їх класифікація. Розглядаються методи та моделі виявлення аномалій, приведено декілька алгоритмів виявлення аномалій. Приведено постановку задачі розробки нової сучасної моделі виявлення вторгнень та вимоги до неї.In this article the analysis of existing IDS is carried out, their classification is given, the advantages and lacks are described. The various methods of detection of anomalies are considered. The mathematical algorithms of detection of abnormal behaviour some IDS are given. The statement of a task of creation of modern model of detection of intrusions and main requirements showed to new model is given

Exploring an agent as an economic insider threat solution

The insider threat is a security problem that is well-known and has a long history, yet it still remains an invisible enemy. Insiders know the security processes and have accesses that allow them to easily cover their tracks. In recent years the idea of monitoring separately for these threats has come into its own. However, the tools currently in use have disadvantages and one of the most effective techniques of human review is costly. This paper explores the development of an intelligent agent that uses already in-place computing material for inference as an inexpensive monitoring tool for insider threats. Design Science Research (DSR) is a methodology used to explore and develop an IT artifact, such as for this intelligent agent research. This methodology allows for a structure that can guide a deep search method for problems that may not be possible to solve or could add to a phenomenological instantiation

Survey of Intrusion Detection Research

The literature holds a great deal of research in the intrusion detection area. Much of this describes the design and implementation of specific intrusion detection systems. While the main focus has been the study of different detection algorithms and methods, there are a number of other issues that are of equal importance to make these systems function well in practice. I believe that the reason that the commercial market does not use many of the ideas described is that there are still too many unresolved issues. This survey focuses on presenting the different issues that must be addressed to build fully functional and practically usable intrusion detection systems (IDSs). It points out the state of the art in each area and suggests important open research issues

Towards Increasing Trust In Expert Evidence Derived From Malware Forensic Tools

Following a series of high profile miscarriages of justice in the UK linked to questionable expert evidence, the post of the Forensic Science Regulator was created in 2008. The main objective of this role is to improve the standard of practitioner competences and forensic procedures. One of the key strategies deployed to achieve this is the push to incorporate a greater level of scientific conduct in the various fields of forensic practice. Currently there is no statutory requirement for practitioners to become accredited to continue working with the Criminal Justice System of England and Wales. However, the Forensic Science Regulator is lobbying the UK Government to make this mandatory. This paper focuses upon the challenge of incorporating a scientific methodology to digital forensic investigations where malicious software (‘malware’) has been identified. One aspect of such a methodology is the approach followed to both select and evaluate the tools used to perform dynamic malware analysis during an investigation. Based on the literature, legal, regulatory and practical needs we derive a set of requirements to address this challenge. We present a framework, called the ‘Malware Analysis Tool Evaluation Framework’ (MATEF), to address this lack of methodology to evaluate software tools used to perform dynamic malware analysis during investigations involving malware and discuss how it meets the derived requirements

A GENERIC ARCHITECTURE FOR INSIDER MISUSE MONITORING IN IT SYSTEMS

Intrusion Detection Systems (IDS) have been widely deployed within many organisations' IT nenvorks to delect network penetration attacks by outsiders and privilege escalation attacks by insiders. However, traditional IDS are ineffective for detecting o f abuse o f legitimate privileges by authorised users within the organisation i.e. the detection of misfeasance. In essence insider IT abuse does not violate system level controls, yet violates acceptable usage policy, business controls, or code of conduct defined by the organisation. However, the acceptable usage policy can vary from one organisation to another, and the acceptability o f user activities can also change depending upon the user(s), application, machine, data, and other contextual conditions associated with the entities involved. The fact that the perpetrators are authorised users and that the insider misuse activities do not violate system level controls makes detection of insider abuse more complicated than detection o f attacks by outsiders. The overall aim o f the research is to determine novel methods by which monitoring and detection may be improved to enable successful detection of insider IT abuse. The discussion begins with a comprehensive investigation o f insider IT misuse, encompassing the breadth and scale of the problem. Consideration is then given to the sufficiency of existing safeguards, with the conclusion that they provide an inadequate basis for detecting many o f the problems. This finding is used as the justification for considering research into alternative approaches. The realisation of the research objective includes the development of a taxonomy for identification o f various levels within the system from which the relevant data associated with each type of misuse can be collected, and formulation of a checklist for identification of applications that requires misfeasor monitoring. Based upon this foundation a novel architecture for monitoring o f insider IT misuse, has been designed. The design offers new analysis procedures to be added, while providing methods to include relevant contextual parameters from dispersed systems for analysis and reference. The proposed system differs from existing IDS in the way that it focuses on detecting contextual misuse of authorised privileges and legitimate operations, rather than detecting exploitation o f network protocols and system level \ailnerabilities. The main concepts of the new architecture were validated through a proof-of-concept prototype system. A number o f case scenarios were used to demonstrate the validity of analysis procedures developed and how the contextual data from dispersed databases can be used for analysis of various types of insider activities. This helped prove that the existing detection technologies can be adopted for detection o f insider IT misuse, and that the research has thus provided valuable contribution to the domain