From Intrusion Detection to Attacker Attribution: A Comprehensive Survey of Unsupervised Methods

Over the last five years there has been an increase in the frequency and diversity of network attacks. This holds true, as more and more organisations admit compromises on a daily basis. Many misuse and anomaly based Intrusion Detection Systems (IDSs) that rely on either signatures, supervised or statistical methods have been proposed in the literature, but their trustworthiness is debatable. Moreover, as this work uncovers, the current IDSs are based on obsolete attack classes that do not reflect the current attack trends. For these reasons, this paper provides a comprehensive overview of unsupervised and hybrid methods for intrusion detection, discussing their potential in the domain. We also present and highlight the importance of feature engineering techniques that have been proposed for intrusion detection. Furthermore, we discuss that current IDSs should evolve from simple detection to correlation and attribution. We descant how IDS data could be used to reconstruct and correlate attacks to identify attackers, with the use of advanced data analytics techniques. Finally, we argue how the present IDS attack classes can be extended to match the modern attacks and propose three new classes regarding the outgoing network communicatio

Revealing Influenced Selected Feature for P2P Botnet Detection

P2P botnet has become a serious security threat for computer networking systems. Botnet attack causes a great financial loss and badly impact the information and communication technology (ICT) system. Current botnet detection mechanisms have limitations and flaws to deal with P2P botnets which famously known for their complexity and scalable attack. Studies show that botnets behavior can be detected based on several detection features. However, some of the feature parameters may not represent botnet behavior and may lead to higher false alarm detection rate. In this paper, we reveal selected feature that influences P2P botnets detection. The result obtained by selecting features shows detection attack rate of 99.74%

Performance of Malware Classification on Machine Learning using Feature Selection

The exponential growth of malware has created a significant threat in our daily lives, which heavily rely on computers running all kinds of software. Malware writers create malicious software by creating new variants, new innovations, new infections and more obfuscated malware by using techniques such as packing and encrypting techniques. Malicious software classification and detection play an important role and a big challenge for cyber security research. Due to the increasing rate of false alarm, the accurate classification and detection of malware is a big necessity issue to be solved. In this research, eight malware family have been classifying according to their family the research provides four feature selection algorithms to select best feature for multiclass classification problem. Comparing. Then find these algorithms top 100 features are selected to performance evaluations. Five machine learning algorithms is compared to find best models. Then frequency distribution of features are find by feature ranking of best model. At last it is said that frequency distribution of every character of API call sequence can be used to classify malware family

Detection of Android Malware using Feature Selection with a Hybrid Genetic Algorithm and Simulated Annealing (SVM and DBN)

Because of the widespread use of the Android operating system and the simplicity with which applications can be created on the Android platform, anyone can easily create malware using pre-made tools. Due to the spread of malware among many helpful applications, Android users are experiencing issues. In this study, we showed how to use permissions gleaned from static analysis to identify Android malware. Utilising support vector machines and deep belief networks, we choose the pertinent features from the set of permissions based on this methodology. The suggested technique increases the effectiveness of Android malware detection

Feature Selection on Permissions, Intents and APIs for Android Malware Detection

Malicious applications pose an enormous security threat to mobile computing devices. Currently 85% of all smartphones run Android, Google’s open-source operating system, making that platform the primary threat vector for malware attacks. Android is a platform that hosts roughly 99% of known malware to date, and is the focus of most research efforts in mobile malware detection due to its open source nature. One of the main tools used in this effort is supervised machine learning. While a decade of work has made a lot of progress in detection accuracy, there is an obstacle that each stream of research is forced to overcome, feature selection, i.e., determining which attributes of Android are most effective as inputs into machine learning models. This dissertation aims to address that problem by providing the community with an exhaustive analysis of the three primary types of Android features used by researchers: Permissions, Intents and API Calls. The intent of the report is not to describe a best performing feature set or a best performing machine learning model, nor to explain why certain Permissions, Intents or API Calls get selected above others, but rather to provide a holistic methodology to help guide feature selection for Android malware detection. The experiments used eleven different feature selection techniques covering filter methods, wrapper methods and embedded methods. Each feature selection technique was applied to seven different datasets based on the seven combinations available of Permissions, Intents and API Calls. Each of those seven datasets are from a base set of 119k Android apps. All of the result sets were then validated against three different machine learning models, Random Forest, SVM and a Neural Net, to test applicability across algorithm type. The experiments show that using a combination of Permissions, Intents and API Calls produced higher accuracy than using any of those alone or in any other combination and that feature selection should be performed on the combined dataset, not by feature type and then combined. The data also shows that, in general, a feature set size of 200 or more attributes is required for optimal results. Finally, the feature selection methods Relief, Correlation-based Feature Selection (CFS) and Recursive Feature Elimination (RFE) using a Neural Net are not satisfactory approaches for Android malware detection work. Based on the proposed methodology and experiments, this research provided insights into feature selection – a significant but often overlooked issue in Android malware detection. We believe the results reported herein is an important step for effective feature evaluation and selection in assisting malware detection especially for datasets with a large number of features. The methodology also has the potential to be applied to similar malware detection tasks or even in broader domains such as pattern recognition

MalDetConv: Automated Behaviour-based Malware Detection Framework Based on Natural Language Processing and Deep Learning Techniques

The popularity of Windows attracts the attention of hackers/cyber-attackers, making Windows devices the primary target of malware attacks in recent years. Several sophisticated malware variants and anti-detection methods have been significantly enhanced and as a result, traditional malware detection techniques have become less effective. This work presents MalBehavD-V1, a new behavioural dataset of Windows Application Programming Interface (API) calls extracted from benign and malware executable files using the dynamic analysis approach. In addition, we present MalDetConV, a new automated behaviour-based framework for detecting both existing and zero-day malware attacks. MalDetConv uses a text processing-based encoder to transform features of API calls into a suitable format supported by deep learning models. It then uses a hybrid of convolutional neural network (CNN) and bidirectional gated recurrent unit (CNN-BiGRU) automatic feature extractor to select high-level features of the API Calls which are then fed to a fully connected neural network module for malware classification. MalDetConv also uses an explainable component that reveals features that contributed to the final classification outcome, helping the decision-making process for security analysts. The performance of the proposed framework is evaluated using our MalBehavD-V1 dataset and other benchmark datasets. The detection results demonstrate the effectiveness of MalDetConv over the state-of-the-art techniques with detection accuracy of 96.10%, 95.73%, 98.18%, and 99.93% achieved while detecting unseen malware from MalBehavD-V1, Allan and John, Brazilian, and Ki-D datasets, respectively. The experimental results show that MalDetConv is highly accurate in detecting both known and zero-day malware attacks on Windows devices

A Novel Hybrid Spotted Hyena-Swarm Optimization (HS-FFO) Framework for Effective Feature Selection in IOT Based Cloud Security Data

Internet of Things (IoT) has gained its major insight in terms of its deployment and applications. Since IoT exhibits more heterogeneous characteristics in transmitting the real time application data, these data are vulnerable to many security threats. To safeguard the data, machine and deep learning based security systems has been proposed. But this system suffers the computational burden that impedes threat detection capability. Hence the feature selection plays an important role in designing the complexity aware IoT systems to defend the security attacks in the system. This paper propose the novel ensemble of spotted hyena with firefly algorithm to choose the best features and minimise the redundant data features that can boost the detection system's computational effectiveness.  Firstly, an effective firefly optimized feature correlation method is developed.  Then, in order to enhance the exploration and search path, operators of fireflies are combined with Spotted Hyena to assist the swarms in leaving the regionally best solutions. The experimentation has been carried out using the different IoT cloud security datasets such as NSL-KDD-99 , UNSW and CIDCC -001 datasets and contrasted with ten cutting-edge feature extraction techniques, like PSO (particle swarm optimization), BAT, Firefly, ACO(Ant Colony Optimization), Improved PSO, CAT, RAT, Spotted Hyena, SHO and  BOC(Bee-Colony Optimization) algorithms. Results demonstrates the proposed hybrid model has achieved the better feature selection mechanism with less convergence  time and aids better for intelligent threat detection system with the high performance of detection