Improving intrusion detection systems using data mining techniques

Recent surveys and studies have shown that cyber-attacks have caused a lot of damage to organisations, governments, and individuals around the world. Although developments are constantly occurring in the computer security field, cyber-attacks still cause damage as they are developed and evolved by hackers. This research looked at some industrial challenges in the intrusion detection area. The research identified two main challenges; the first one is that signature-based intrusion detection systems such as SNORT lack the capability of detecting attacks with new signatures without human intervention. The other challenge is related to multi-stage attack detection, it has been found that signature-based is not efficient in this area. The novelty in this research is presented through developing methodologies tackling the mentioned challenges. The first challenge was handled by developing a multi-layer classification methodology. The first layer is based on decision tree, while the second layer is a hybrid module that uses two data mining techniques; neural network, and fuzzy logic. The second layer will try to detect new attacks in case the first one fails to detect. This system detects attacks with new signatures, and then updates the SNORT signature holder automatically, without any human intervention. The obtained results have shown that a high detection rate has been obtained with attacks having new signatures. However, it has been found that the false positive rate needs to be lowered. The second challenge was approached by evaluating IP information using fuzzy logic. This approach looks at the identity of participants in the traffic, rather than the sequence and contents of the traffic. The results have shown that this approach can help in predicting attacks at very early stages in some scenarios. However, it has been found that combining this approach with a different approach that looks at the sequence and contents of the traffic, such as event- correlation, will achieve a better performance than each approach individually

Cost benefits of using machine learning features in NIDS for cyber security in UK small medium enterprises (SME)

Cyber security has made an impact and has challenged Small and Medium Enterprises (SMEs) in their approaches towards how they protect and secure data. With an increase in more wired and wireless connections and devices on SME networks, unpredictable malicious activities and interruptions have risen. Finding the harmony between the advancement of technology and costs has always been a balancing act particularly in convincing the finance directors of these SMEs to invest in capital towards their IT infrastructure. This paper looks at various devices that currently are in the market to detect intrusions and look at how these devices handle prevention strategies for SMEs in their working environment both at home and in the office, in terms of their credibility in handling zero-day attacks against the costs of achieving so. The experiment was set up during the 2020 pandemic referred to as COVID-19 when the world experienced an unprecedented event of large scale. The operational working environment of SMEs reflected the context when the UK went into lockdown. Pre-pandemic would have seen this experiment take full control within an operational office environment; however, COVID-19 times has pushed us into a corner to evaluate every aspect of cybersecurity from the office and keeping the data safe within the home environment. The devices chosen for this experiment were OpenSource such as SNORT and pfSense to detect activities within the home environment, and Cisco, a commercial device, set up within an SME network. All three devices operated in a live environment within the SME network structure with employees being both at home and in the office. All three devices were observed from the rules they displayed, their costs and machine learning techniques integrated within them. The results revealed these aspects to be important in how they identified zero-day attacks. The findings showed that OpenSource devices whilst free to download, required a high level of expertise in personnel to implement and embed machine learning rules into the business solution even for staff working from home. However, when using Cisco, the price reflected the buy-in into this expertise and Cisco’s mainframe network, to give up-to-date information on cyber-attacks. The requirements of the UK General Data Protection Regulations Act (GDPR) were also acknowledged as part of the broader framework of the study. Machine learning techniques such as anomaly-based intrusions did show better detection through a commercially subscription-based model for support from Cisco compared to that of the OpenSource model which required internal expertise in machine learning. A cost model was used to compare the outcome of SMEs’ decision making, in getting the right framework in place in securing their data. In conclusion, finding a balance between IT expertise and costs of products that are able to help SMEs protect and secure their data will benefit the SMEs from using a more intelligent controlled environment with applied machine learning techniques, and not compromising on costs.</p

Feature Subset Selection in Intrusion Detection Using Soft Computing Techniques

Intrusions on computer network systems are major security issues these days. Therefore, it is of utmost importance to prevent such intrusions. The prevention of such intrusions is entirely dependent on their detection that is a main part of any security tool such as Intrusion Detection System (IDS), Intrusion Prevention System (IPS), Adaptive Security Alliance (ASA), checkpoints and firewalls. Therefore, accurate detection of network attack is imperative. A variety of intrusion detection approaches are available but the main problem is their performance, which can be enhanced by increasing the detection rates and reducing false positives. Such weaknesses of the existing techniques have motivated the research presented in this thesis. One of the weaknesses of the existing intrusion detection approaches is the usage of a raw dataset for classification but the classifier may get confused due to redundancy and hence may not classify correctly. To overcome this issue, Principal Component Analysis (PCA) has been employed to transform raw features into principal features space and select the features based on their sensitivity. The sensitivity is determined by the values of eigenvalues. The recent approaches use PCA to project features space to principal feature space and select features corresponding to the highest eigenvalues, but the features corresponding to the highest eigenvalues may not have the optimal sensitivity for the classifier due to ignoring many sensitive features. Instead of using traditional approach of selecting features with the highest eigenvalues such as PCA, this research applied a Genetic Algorithm (GA) to search the principal feature space that offers a subset of features with optimal sensitivity and the highest discriminatory power. Based on the selected features, the classification is performed. The Support Vector Machine (SVM) and Multilayer Perceptron (MLP) are used for classification purpose due to their proven ability in classification. This research work uses the Knowledge Discovery and Data mining (KDD) cup dataset, which is considered benchmark for evaluating security detection mechanisms. The performance of this approach was analyzed and compared with existing approaches. The results show that proposed method provides an optimal intrusion detection mechanism that outperforms the existing approaches and has the capability to minimize the number of features and maximize the detection rates

Feature Subset Selection in Intrusion Detection Using Soft Computing Techniques

Intrusions on computer network systems are major security issues these days. Therefore, it is of utmost importance to prevent such intrusions. The prevention of such intrusions is entirely dependent on their detection that is a main part of any security tool such as Intrusion Detection System (IDS), Intrusion Prevention System (IPS), Adaptive Security Alliance (ASA), checkpoints and firewalls. Therefore, accurate detection of network attack is imperative. A variety of intrusion detection approaches are available but the main problem is their performance, which can be enhanced by increasing the detection rates and reducing false positives. Such weaknesses of the existing techniques have motivated the research presented in this thesis. One of the weaknesses of the existing intrusion detection approaches is the usage of a raw dataset for classification but the classifier may get confused due to redundancy and hence may not classify correctly. To overcome this issue, Principal Component Analysis (PCA) has been employed to transform raw features into principal features space and select the features based on their sensitivity. The sensitivity is determined by the values of eigenvalues. The recent approaches use PCA to project features space to principal feature space and select features corresponding to the highest eigenvalues, but the features corresponding to the highest eigenvalues may not have the optimal sensitivity for the classifier due to ignoring many sensitive features. Instead of using traditional approach of selecting features with the highest eigenvalues such as PCA, this research applied a Genetic Algorithm (GA) to search the principal feature space that offers a subset of features with optimal sensitivity and the highest discriminatory power. Based on the selected features, the classification is performed. The Support Vector Machine (SVM) and Multilayer Perceptron (MLP) are used for classification purpose due to their proven ability in classification. This research work uses the Knowledge Discovery and Data mining (KDD) cup dataset, which is considered benchmark for evaluating security detection mechanisms. The performance of this approach was analyzed and compared with existing approaches. The results show that proposed method provides an optimal intrusion detection mechanism that outperforms the existing approaches and has the capability to minimize the number of features and maximize the detection rates

A taxonomy of attacks and a survey of defence mechanisms for semantic social engineering attacks

Social engineering is used as an umbrella term for a broad spectrum of computer exploitations that employ a variety of attack vectors and strategies to psychologically manipulate a user. Semantic attacks are the specific type of social engineering attacks that bypass technical defences by actively manipulating object characteristics, such as platform or system applications, to deceive rather than directly attack the user. Commonly observed examples include obfuscated URLs, phishing emails, drive-by downloads, spoofed web- sites and scareware to name a few. This paper presents a taxonomy of semantic attacks, as well as a survey of applicable defences. By contrasting the threat landscape and the associated mitigation techniques in a single comparative matrix, we identify the areas where further research can be particularly beneficial

Robust filtering schemes for machine learning systems to defend Adversarial Attack

Robust filtering schemes for machine learning systems to defend Adversarial Attac