Improving intrusion detection systems using data mining techniques

Recent surveys and studies have shown that cyber-attacks have caused a lot of damage to organisations, governments, and individuals around the world. Although developments are constantly occurring in the computer security field, cyber-attacks still cause damage as they are developed and evolved by hackers. This research looked at some industrial challenges in the intrusion detection area. The research identified two main challenges; the first one is that signature-based intrusion detection systems such as SNORT lack the capability of detecting attacks with new signatures without human intervention. The other challenge is related to multi-stage attack detection, it has been found that signature-based is not efficient in this area. The novelty in this research is presented through developing methodologies tackling the mentioned challenges. The first challenge was handled by developing a multi-layer classification methodology. The first layer is based on decision tree, while the second layer is a hybrid module that uses two data mining techniques; neural network, and fuzzy logic. The second layer will try to detect new attacks in case the first one fails to detect. This system detects attacks with new signatures, and then updates the SNORT signature holder automatically, without any human intervention. The obtained results have shown that a high detection rate has been obtained with attacks having new signatures. However, it has been found that the false positive rate needs to be lowered. The second challenge was approached by evaluating IP information using fuzzy logic. This approach looks at the identity of participants in the traffic, rather than the sequence and contents of the traffic. The results have shown that this approach can help in predicting attacks at very early stages in some scenarios. However, it has been found that combining this approach with a different approach that looks at the sequence and contents of the traffic, such as event- correlation, will achieve a better performance than each approach individually

A Collaborative Architecture for Distributed Intrusion Detection System based on Lightweight Modules

A variety of intrusion prevention techniques, such as user authentication (e.g.: using passwords), avoidance of programming errors, and information protection, have been used to protect computer systems. However, intrusion prevention alone is not sufficient to protect our systems, as those systems become ever more complex with the rapid growth and expansion of Internet technology and local network systems. Moreover, programming errors, firewall configuration errors, and ambiguous or undefined security policies add to the system’s complexity. An Intrusion Detection System (IDS) is therefore needed as another layer to protect computer systems. The IDS is one of the most important techniques of information dynamic security technology. It is defined as a process of monitoring the events occurring in a computer system or network and analyzing them to differentiate between normal activities of the system and behaviours that can be classified as suspicious or intrusive. Current Intrusion Detection Systems have several known shortcomings, such as: low accuracy (registering high False Positives and False Negatives); low real-time performance (processing a large amount of traffic in real time); limited scalability (storing a large number of user profiles and attack signatures); an inability to detect new attacks (recognizing new attacks when they are launched for the first time); and weak system-reactive capabilities (efficiency of response). This makes the area of IDS an attractive research field. In recent years, researchers have investigated techniques such as artificial intelligence, autonomous agents, and distributed systems for detecting intrusion in network environments. This thesis presents a novel IDS distributed architecture – Collaborative Distributed Intrusion Detection System (C-dIDS), based on lightweight IDS modules – that integrates two main concepts in order to improve IDS performance and the scalability: lightweight IDS and collaborative architecture. To accomplish the first concept, lightweight IDS, we apply two different approaches: a features selection approach and an IDS classification scheme. In the first approach, each detector (IDS module) uses smaller amounts of data in the detection process by applying a novel features selection approach called the Fuzzy Enhanced Support Vector Decision Function (Fuzzy ESVDF). This approach improves the system scalability in terms of reducing the number of needed features without degrading the overall system performance. The second approach uses a new IDS classification scheme. The proposed IDS classification scheme employs multiple specialized detectors in each layer of the TCP/IP network model. This helps collecting efficient and useful information for dIDS, increasing the system’s ability to detect different attack types and reducing the system’s scalability. The second concept uses a novel architecture for dIDS called Collaborative Distributed Intrusion Detection System (C-dIDS) to integrate these different specialized detectors (IDS modules) that are distributed on different points in the network. This architecture is a single-level hierarchy dIDS with a non-central analyzer. To make the detection decision for a specific IDS module in the system, this module must collaborate with the previous IDS module (host) in the lower level of the hierarchy only. Collaborating with other IDS modules improves the overall system accuracy without creating a heavy system overload. Also, this architecture avoids both single point of failure and scalability bottleneck problems. Integration of the two main concepts, lightweight IDS and a distributed collaborative architecture, has shown very good results and has addressed many IDS limitations

ESTUDIO COMPARATIVO DE TÉCNICAS DE ENTRENAMIENTO Y CLASIFICACIÓN EN SISTEMAS DE DETECCIÓN DE INSTRUSOS (IDS), BASADOS EN ANOMALIAS DE RED.

Maestría en Ingeniería (Énfasis en Redes y Software)The main motivation of this investigation was the implementation of the Draper method applied to intrusion detection systems in different training and classification techniques in order to identify the best intrusion detection model with the objective of improving detection rates of attacks in computer network systems, using a procedure of selection of characteristics and different methods of algorithms of unsupervised trainings, in this case was used the technique INFO.GAIN identifying that the number of optimal characteristics is 15. Consequently, a neural network using a non-supervised learning algorithm (GHSOM, RANDOM FOREST, BAYESIAN NETWORKS, NAIVE BAYES, C4.5, LOGISTIC, PART AND NBTREE) for the purpose of classifying bi-class traffic automatically. obtained the best technique of training and classification using the selection technique In INFO.GAIN with 15 characteristics and cross validation 10 pligues, was the RANDOM FOREST technique.La principal motivación de esta investigación ha sido la implementación del método Draper aplicado a los sistemas de detección de intrusos en distintas técnicas de entrenamiento y clasificación con el propósito de identificar el mejor modelo de detección de intrusiones con el objetivo de mejorar las tasas de detección de ataques en sistemas de redes computacionales, utilizando un procedimiento de selección de características y distintos métodos de algoritmos de entrenamientos no supervisados, en este caso se utilizó la técnica INFO.GAIN identificando que el número de características óptimo es 15. En consecuencia, se entrenó una red neuronal que utilizan un algoritmo de aprendizaje no supervisado (GHSOM, RANDOM FOREST, REDES BAYESIANAS, NAIVE BAYES, C4.5,LOGISTIC, PART Y NBTREE ), con el propósito de clasificar el tráfico bi-clase de forma automática, Como resultado se obtuvo que la mejor técnica de entrenamiento y clasificación utilizando la técnica de selección INFO.GAIN a 15 características y validación cruzada 10 pligues, fue la técnica RANDOM FOREST

Cyber defensive capacity and capability::A perspective from the financial sector of a small state

This thesis explores ways in which the financial sectors of small states are able todefend themselves against ever-growing cyber threats, as well as ways these states can improve their cyber defense capability in order to withstand current andfuture attacks. To date, the context of small states in general is understudied. This study presents the challenges faced by financial sectors in small states with regard to withstanding cyberattacks. This study applies a mixed method approach through the use of various surveys, brainstorming sessions with financial sector focus groups, interviews with critical infrastructure stakeholders, a literature review, a comparative analysis of secondary data and a theoretical narrative review. The findings suggest that, for the Aruban financial sector, compliance is important, as with minimal drivers, precautionary behavior is significant. Countermeasures of formal, informal, and technical controls need to be in place. This study indicates the view that defending a small state such as Aruba is challenging, yet enough economic indicators indicate it not being outside the realm of possibility. On a theoretical level, this thesis proposes a conceptual “whole-of-cyber” model inspired by military science and the VSM (Viable Systems Model). The concept of fighting power components and governance S4 function form cyber defensive capacity’s shield and capability. The “whole-of-cyber” approach may be a good way to compensate for the lack of resources of small states. Collaboration may be an only out, as the fastest-growing need will be for advanced IT skillsets

Combining SOA and BPM Technologies for Cross-System Process Automation

This paper summarizes the results of an industry case study that introduced a cross-system business process automation solution based on a combination of SOA and BPM standard technologies (i.e., BPMN, BPEL, WSDL). Besides discussing major weaknesses of the existing, custom-built, solution and comparing them against experiences with the developed prototype, the paper presents a course of action for transforming the current solution into the proposed solution. This includes a general approach, consisting of four distinct steps, as well as specific action items that are to be performed for every step. The discussion also covers language and tool support and challenges arising from the transformation