AI Solutions for MDS: Artificial Intelligence Techniques for Misuse Detection and Localisation in Telecommunication Environments

This report considers the application of Articial Intelligence (AI) techniques to the problem of misuse detection and misuse localisation within telecommunications environments. A broad survey of techniques is provided, that covers inter alia rule based systems, model-based systems, case based reasoning, pattern matching, clustering and feature extraction, articial neural networks, genetic algorithms, arti cial immune systems, agent based systems, data mining and a variety of hybrid approaches. The report then considers the central issue of event correlation, that is at the heart of many misuse detection and localisation systems. The notion of being able to infer misuse by the correlation of individual temporally distributed events within a multiple data stream environment is explored, and a range of techniques, covering model based approaches, `programmed' AI and machine learning paradigms. It is found that, in general, correlation is best achieved via rule based approaches, but that these suffer from a number of drawbacks, such as the difculty of developing and maintaining an appropriate knowledge base, and the lack of ability to generalise from known misuses to new unseen misuses. Two distinct approaches are evident. One attempts to encode knowledge of known misuses, typically within rules, and use this to screen events. This approach cannot generally detect misuses for which it has not been programmed, i.e. it is prone to issuing false negatives. The other attempts to `learn' the features of event patterns that constitute normal behaviour, and, by observing patterns that do not match expected behaviour, detect when a misuse has occurred. This approach is prone to issuing false positives, i.e. inferring misuse from innocent patterns of behaviour that the system was not trained to recognise. Contemporary approaches are seen to favour hybridisation, often combining detection or localisation mechanisms for both abnormal and normal behaviour, the former to capture known cases of misuse, the latter to capture unknown cases. In some systems, these mechanisms even work together to update each other to increase detection rates and lower false positive rates. It is concluded that hybridisation offers the most promising future direction, but that a rule or state based component is likely to remain, being the most natural approach to the correlation of complex events. The challenge, then, is to mitigate the weaknesses of canonical programmed systems such that learning, generalisation and adaptation are more readily facilitated

Machine Learning in Wireless Sensor Networks: Algorithms, Strategies, and Applications

Wireless sensor networks monitor dynamic environments that change rapidly over time. This dynamic behavior is either caused by external factors or initiated by the system designers themselves. To adapt to such conditions, sensor networks often adopt machine learning techniques to eliminate the need for unnecessary redesign. Machine learning also inspires many practical solutions that maximize resource utilization and prolong the lifespan of the network. In this paper, we present an extensive literature review over the period 2002-2013 of machine learning methods that were used to address common issues in wireless sensor networks (WSNs). The advantages and disadvantages of each proposed algorithm are evaluated against the corresponding problem. We also provide a comparative guide to aid WSN designers in developing suitable machine learning solutions for their specific application challenges.Comment: Accepted for publication in IEEE Communications Surveys and Tutorial

Data mining approaches for detecting intrusion using UNIX process execution traces

Intrusion detection systems help computer systems prepare for and deal with malicious attacks. They collect information from a variety of systems and network sources, then analyze the information for signs of intrusion and misuse. A variety of techniques have been employed to analyze the information from traditional statistical methods to new emerged data mining approaches. In this thesis, we describe several algorithms designed for this task, including neural networks, rule induction with C4.5, and Rough sets methods. We compare the classification accuracy of the various methods in a set of UNIX process execution traces. We used two kinds of evaluation methods. The first evaluation criterion characterizes performances over a set of individual classifications in terms of average testing accuracy rate. The second measures the true and false positive rates of the classification output over certain threshold. Experiments were run on data sets of system calls created by synthetic sendmail programs. There were two types of representation methods used. Different combinations of parameters were tested during the experiment. Results indicate that for a wide range of conditions, Rough sets have higher classification accuracy than that of Neural networks and C4.5. In terms of true and false positive evaluations, Rough sets and Neural networks turned out to be better than C4.5

PRECEDENT-FREE FAULT LOCALIZATION AND DIAGNOSIS FOR HIGH SPEED TRAIN DRIVE SYSTEMS

In this paper, a framework for localization of sources of unprecedented faults in the drive train system of high speed trains is presented. The framework utilizes distributed anomaly detection, with anomaly detectors based on the recently introduced Growing Structure Multiple Model Systems (GSMMS) models. Physics based models of the drive system and its pertinent subsystems were derived and were calibrated using data collected over several actual trips on a high speed train. Simulation results demonstrate the ability to localize faults within various parts of the drive train system without the need for models of the underlying faults. In addition, traditional model based diagnosis was utilized for positive identification of faults, with signals emitted by the systems in the presence of those faults being available for modeling and subsequent recognition of faulty behavior

BIOLOGICAL INSPIRED INTRUSION PREVENTION AND SELF-HEALING SYSTEM FOR CRITICAL SERVICES NETWORK

With the explosive development of the critical services network systems and Internet, the need for networks security systems have become even critical with the enlargement of information technology in everyday life. Intrusion Prevention System (IPS) provides an in-line mechanism focus on identifying and blocking malicious network activity in real time. This thesis presents new intrusion prevention and self-healing system (SH) for critical services network security. The design features of the proposed system are inspired by the human immune system, integrated with pattern recognition nonlinear classification algorithm and machine learning. Firstly, the current intrusions preventions systems, biological innate and adaptive immune systems, autonomic computing and self-healing mechanisms are studied and analyzed. The importance of intrusion prevention system recommends that artificial immune systems (AIS) should incorporate abstraction models from innate, adaptive immune system, pattern recognition, machine learning and self-healing mechanisms to present autonomous IPS system with fast and high accurate detection and prevention performance and survivability for critical services network system. Secondly, specification language, system design, mathematical and computational models for IPS and SH system are established, which are based upon nonlinear classification, prevention predictability trust, analysis, self-adaptation and self-healing algorithms. Finally, the validation of the system carried out by simulation tests, measuring, benchmarking and comparative studies. New benchmarking metrics for detection capabilities, prevention predictability trust and self-healing reliability are introduced as contributions for the IPS and SH system measuring and validation. Using the software system, design theories, AIS features, new nonlinear classification algorithm, and self-healing system show how the use of presented systems can ensure safety for critical services networks and heal the damage caused by intrusion. This autonomous system improves the performance of the current intrusion prevention system and carries on system continuity by using self-healing mechanism

Vulnerability analysis of AIS-based intrusion detection systems using genetic and evolutionary hackers

In this thesis, an overview of current intrusion detection methods, evolutionary computation, and immunity-based intrusion detection systems (IDSs) is presented. An application named Genetic Interactive Teams for Intrusion Detection Design and Analysis (GENERTIA) is introduced which uses genetic algorithm (GA)-based hackers known as a red team in order to find vulnerabilities, or holes, in an artificial immune system (AlS)-based IDS. GENERTIA also uses a GA-based blue team in order to repair the holes it finds. The performance of the GA-based hackers is tested and measured according to the number of distinct holes that it finds. The GA-based red team�s behavior is then compared to that of 12 variations of the particle swarm optimization (PSO)-based red team named SWO, SW0+, SW1, SW2, SW3, SW4, CCSWO, CCSW0+, CCSW1, CCSW2, CCSW3, and CCSW4. Each variant of the PSO-based red team differs in terms of the way that it searches for holes in an IDS. Through this test, it is determined that none of the red teams based on PSO perform as well as the one based on a GA. However, two of the twelve PSO-based red teams, CCSW4 and SW0+, provide hole finding capabilities closest to that of the GA. In addition to the ability of the different red teams to find holes in an AlS-based IDS, the search behaviors of the GA-based hackers, PSO-based hackers that use a variable called a constriction coefficient, and PSO-based hackers that do not use the coefficient are compared. The results of this comparison show that it may be possible to implement a red team based on a hybrid �genetic swarm� that improves upon the performance of both the GA- and PSO-based red teams

Intelligent network intrusion detection using an evolutionary computation approach

With the enormous growth of users\u27 reliance on the Internet, the need for secure and reliable computer networks also increases. Availability of effective automatic tools for carrying out different types of network attacks raises the need for effective intrusion detection systems. Generally, a comprehensive defence mechanism consists of three phases, namely, preparation, detection and reaction. In the preparation phase, network administrators aim to find and fix security vulnerabilities (e.g., insecure protocol and vulnerable computer systems or firewalls), that can be exploited to launch attacks. Although the preparation phase increases the level of security in a network, this will never completely remove the threat of network attacks. A good security mechanism requires an Intrusion Detection System (IDS) in order to monitor security breaches when the prevention schemes in the preparation phase are bypassed. To be able to react to network attacks as fast as possible, an automatic detection system is of paramount importance. The later an attack is detected, the less time network administrators have to update their signatures and reconfigure their detection and remediation systems. An IDS is a tool for monitoring the system with the aim of detecting and alerting intrusive activities in networks. These tools are classified into two major categories of signature-based and anomaly-based. A signature-based IDS stores the signature of known attacks in a database and discovers occurrences of attacks by monitoring and comparing each communication in the network against the database of signatures. On the other hand, mechanisms that deploy anomaly detection have a model of normal behaviour of system and any significant deviation from this model is reported as anomaly. This thesis aims at addressing the major issues in the process of developing signature based IDSs. These are: i) their dependency on experts to create signatures, ii) the complexity of their models, iii) the inflexibility of their models, and iv) their inability to adapt to the changes in the real environment and detect new attacks. To meet the requirements of a good IDS, computational intelligence methods have attracted considerable interest from the research community. This thesis explores a solution to automatically generate compact rulesets for network intrusion detection utilising evolutionary computation techniques. The proposed framework is called ESR-NID (Evolving Statistical Rulesets for Network Intrusion Detection). Using an interval-based structure, this method can be deployed for any continuous-valued input data. Therefore, by choosing appropriate statistical measures (i.e. continuous-valued features) of network trafc as the input to ESRNID, it can effectively detect varied types of attacks since it is not dependent on the signatures of network packets. In ESR-NID, several innovations in the genetic algorithm were developed to keep the ruleset small. A two-stage evaluation component in the evolutionary process takes the cooperation of rules into consideration and results into very compact, easily understood rulesets. The effectiveness of this approach is evaluated against several sources of data for both detection of normal and abnormal behaviour. The results are found to be comparable to those achieved using other machine learning methods from both categories of GA-based and non-GA-based methods. One of the significant advantages of ESR-NIS is that it can be tailored to specific problem domains and the characteristics of the dataset by the use of different fitness and performance functions. This makes the system a more flexible model compared to other learning techniques. Additionally, an IDS must adapt itself to the changing environment with the least amount of configurations. ESR-NID uses an incremental learning approach as new flow of traffic become available. The incremental learning approach benefits from less required storage because it only keeps the generated rules in its database. This is in contrast to the infinitely growing size of repository of raw training data required for traditional learning

Development of a Methodology for Condition-Based Maintenance in a Large-Scale Application Field

This paper describes a methodology, developed by the authors, for condition monitoring and diagnostics of several critical components in the large-scale applications with machines. For industry, the main target of condition monitoring is to prevent the machine stopping suddenly and thus avoid economic losses due to lack of production. Once the target is reached at a local level, usually through an R&D project, the extension to a large-scale market gives rise to new goals, such as low computational costs for analysis, easily interpretable results by local technicians, collection of data from worldwide machine installations, and the development of historical datasets to improve methodology, etc. This paper details an approach to condition monitoring, developed together with a multinational corporation, that covers all the critical points mentioned above

Two-tier Intrusion Detection System for Mobile Ad Hoc Networks

Nowadays, a commonly used wireless network (i.e. Wi-Fi) operates with the aid of a fixed infrastructure (i.e. an access point) to facilitate communication between nodes when they roam from one location to another. The need for such a fixed supporting infrastructure limits the adaptability of the wireless network, especially in situations where the deployment of such an infrastructure is impractical. In addition, Wi-Fi limits nodes' communication as it only provides facility for mobile nodes to send and receive information, but not reroute the information across the network. Recent advancements in computer network introduced a new wireless network, known as a Mobile Ad Hoc Network (MANET), to overcome these limitations. MANET has a set of unique characteristics that make it different from other kind of wireless networks. Often referred as a peer to peer network, such a network does not have any fixed topology, thus nodes are free to roam anywhere, and could join or leave the network anytime they desire. Its ability to be setup without the need of any infrastructure is very useful, especially in geographically constrained environments such as in a military battlefield or a disaster relief operation. In addition, through its multi hop routing facility, each node could function as a router, thus communication between nodes could be made available without the need of a supporting fixed router or an access point. However, these handy facilities come with big challenges, especially in dealing with the security issues. This research aims to address MANET security issues by proposing a novel intrusion detection system that could be used to complement existing prevention mechanisms that have been proposed to secure such a network. A comprehensive analysis of attacks and the existing security measures proved that there is a need for an Intrusion Detection System (IDS) to protect MANETs against security threats. The analysis also suggested that the existing IDS proposed for MANET are not immune against a colluding blackmail attack due to the nature of such a network that comprises autonomous and anonymous nodes. The IDS architecture as proposed in this study utilises trust relationships between nodes to overcome this nodes' anonymity issue. Through a friendship mechanism, the problems of false accusations and false alarms caused by blackmail attackers in global detection and response mechanisms could be eliminated. The applicability of the friendship concept as well as other proposed mechanisms to solve MANET IDS related issues have been validated through a set of simulation experiments. Several MANET settings, which differ from each other based on the network's density level, the number of initial trusted friends owned by each node, and the duration of the simulation times, have been used to study the effects of such factors towards the overall performance of the proposed IDS framework. The results obtained from the experiments proved that the proposed concepts are capable to at least minimise i f not fully eliminate the problem currently faced in MANET IDS