AI Solutions for MDS: Artificial Intelligence Techniques for Misuse Detection and Localisation in Telecommunication Environments

This report considers the application of Articial Intelligence (AI) techniques to the problem of misuse detection and misuse localisation within telecommunications environments. A broad survey of techniques is provided, that covers inter alia rule based systems, model-based systems, case based reasoning, pattern matching, clustering and feature extraction, articial neural networks, genetic algorithms, arti cial immune systems, agent based systems, data mining and a variety of hybrid approaches. The report then considers the central issue of event correlation, that is at the heart of many misuse detection and localisation systems. The notion of being able to infer misuse by the correlation of individual temporally distributed events within a multiple data stream environment is explored, and a range of techniques, covering model based approaches, `programmed' AI and machine learning paradigms. It is found that, in general, correlation is best achieved via rule based approaches, but that these suffer from a number of drawbacks, such as the difculty of developing and maintaining an appropriate knowledge base, and the lack of ability to generalise from known misuses to new unseen misuses. Two distinct approaches are evident. One attempts to encode knowledge of known misuses, typically within rules, and use this to screen events. This approach cannot generally detect misuses for which it has not been programmed, i.e. it is prone to issuing false negatives. The other attempts to `learn' the features of event patterns that constitute normal behaviour, and, by observing patterns that do not match expected behaviour, detect when a misuse has occurred. This approach is prone to issuing false positives, i.e. inferring misuse from innocent patterns of behaviour that the system was not trained to recognise. Contemporary approaches are seen to favour hybridisation, often combining detection or localisation mechanisms for both abnormal and normal behaviour, the former to capture known cases of misuse, the latter to capture unknown cases. In some systems, these mechanisms even work together to update each other to increase detection rates and lower false positive rates. It is concluded that hybridisation offers the most promising future direction, but that a rule or state based component is likely to remain, being the most natural approach to the correlation of complex events. The challenge, then, is to mitigate the weaknesses of canonical programmed systems such that learning, generalisation and adaptation are more readily facilitated

Survey of Intrusion Detection Research

The literature holds a great deal of research in the intrusion detection area. Much of this describes the design and implementation of specific intrusion detection systems. While the main focus has been the study of different detection algorithms and methods, there are a number of other issues that are of equal importance to make these systems function well in practice. I believe that the reason that the commercial market does not use many of the ideas described is that there are still too many unresolved issues. This survey focuses on presenting the different issues that must be addressed to build fully functional and practically usable intrusion detection systems (IDSs). It points out the state of the art in each area and suggests important open research issues

ANOMALY NETWORK INTRUSION DETECTION SYSTEM BASED ON DISTRIBUTED TIME-DELAY NEURAL NETWORK (DTDNN)

In this research, a hierarchical off-line anomaly network intrusion detection system based on Distributed Time-Delay Artificial Neural Network is introduced. This research aims to solve a hierarchical multi class problem in which the type of attack (DoS, U2R, R2L and Probe attack) detected by dynamic neural network. The results indicate that dynamic neural nets (Distributed Time-Delay Artificial Neural Network) can achieve a high detection rate, where the overall accuracy classification rate average is equal to 97.24%

FORTES: Forensic Information Flow Analysis of Business Processes

Nearly 70% of all business processes in use today rely on automated workflow systems for their execution. Despite the growing expenses in the design of advanced tools for secure and compliant deployment of workflows, an exponential growth of dependability incidents persists. Concepts beyond access control focusing on information flow control offer new paradigms to design security mechanisms for reliable and secure IT-based workflows. This talk presents FORTES, an approach for the forensic analysis of information flow properties. FORTES claims that information flow control can be made usable as a core of an audit-control system. For this purpose, it reconstructs workflow models from secure log files (i.e. execution traces) and, applying security policies, analyzes the information flows to distinguish security relevant from security irrelevant information flows. FORTES thus cannot prevent security policy violations, but by detecting them with well-founded analysis, improve the precision of audit controls and the generated certificates

Feature selection and visualization techniques for network anomaly detector

Intrusion detection systems have been widely used as burglar alarms in the computer security field. There are two major types of detection techniques: misuse detection and anomaly detection. Although misuse detection can detect known attacks with lower false positive rate, anomaly detection is capable of detecting any new or varied attempted intrusion as long as the attempted intrusions disturb the normal states of the systems. The network anomaly detector is employed to monitor a segment of network for any suspicious activities based on the sniffered network traffic. The fast speed of network and wide use of encryption techniques make it almost unpractical to read payload information for the network anomaly detector. This work tries to answer the question: What are the best features for network anomaly detector? The main experiment data sets are from 1999 DARPA Lincoln Library off-line intrusion evaluation project since it is still the most comprehensive public benchmark data up to today. Firstly, 43 features of different levels and protocols are defined. Using the first three weeks as training data and last two weeks as testing data, the performance of the features are testified by using 5 different classifiers. Secondly, the feasibility of feature selection is investigated by employing some filter and wrapper techniques such as Correlation Feature Selection, etc. Thirdly, the effect of changing overlap and time window for the network anomaly detector is investigated. At last, GGobi and Mineset are utilized to visualize intrusion detections to save time and effort for system administrators. The results show the capability of our features is not limited to probing attacks and denial of service attacks. They can also detect remote to local attacks and backdoors. The feature selection techniques successfully reduce the dimensionality of the features from 43 to 10 without performance degrading. The three dimensional visualization pictures provide a straightforward view of normal network traffic and malicious attacks. The time plot of key features can be used to aid system administrators to quickly locate the possible intrusions

A Validity-Based Approach for Feature Selection in Intrusion Detection Systems

Intrusion detection systems are tools that detect and remedy the presence of malicious activities. Intrusion detection systems face many challenges in terms of accurate analysis and evaluation. One such challenge is the involvement of many features during analysis, which leads to high data volume and ultimately excessive computational overhead. This research surrounds the development of a new intrusion detection system by employing an entropy-based measure called v-measure to select significant features and reduce dimensionality. After the development of the intrusion detection system, this feature reduction technique was tested on public datasets by applying machine learning classifiers such as Decision Tree, Random Forest, and AdaBoost algorithms. We have compared the results of the features selected with other feature selection techniques for correct classification of attacks. The findings demonstrated dimension and data volume reduction while maintaining low false positive rate, low false negative rate, and high detection rate