From Intrusion Detection to Attacker Attribution: A Comprehensive Survey of Unsupervised Methods

Over the last five years there has been an increase in the frequency and diversity of network attacks. This holds true, as more and more organisations admit compromises on a daily basis. Many misuse and anomaly based Intrusion Detection Systems (IDSs) that rely on either signatures, supervised or statistical methods have been proposed in the literature, but their trustworthiness is debatable. Moreover, as this work uncovers, the current IDSs are based on obsolete attack classes that do not reflect the current attack trends. For these reasons, this paper provides a comprehensive overview of unsupervised and hybrid methods for intrusion detection, discussing their potential in the domain. We also present and highlight the importance of feature engineering techniques that have been proposed for intrusion detection. Furthermore, we discuss that current IDSs should evolve from simple detection to correlation and attribution. We descant how IDS data could be used to reconstruct and correlate attacks to identify attackers, with the use of advanced data analytics techniques. Finally, we argue how the present IDS attack classes can be extended to match the modern attacks and propose three new classes regarding the outgoing network communicatio

MalDetConv: Automated Behaviour-based Malware Detection Framework Based on Natural Language Processing and Deep Learning Techniques

The popularity of Windows attracts the attention of hackers/cyber-attackers, making Windows devices the primary target of malware attacks in recent years. Several sophisticated malware variants and anti-detection methods have been significantly enhanced and as a result, traditional malware detection techniques have become less effective. This work presents MalBehavD-V1, a new behavioural dataset of Windows Application Programming Interface (API) calls extracted from benign and malware executable files using the dynamic analysis approach. In addition, we present MalDetConV, a new automated behaviour-based framework for detecting both existing and zero-day malware attacks. MalDetConv uses a text processing-based encoder to transform features of API calls into a suitable format supported by deep learning models. It then uses a hybrid of convolutional neural network (CNN) and bidirectional gated recurrent unit (CNN-BiGRU) automatic feature extractor to select high-level features of the API Calls which are then fed to a fully connected neural network module for malware classification. MalDetConv also uses an explainable component that reveals features that contributed to the final classification outcome, helping the decision-making process for security analysts. The performance of the proposed framework is evaluated using our MalBehavD-V1 dataset and other benchmark datasets. The detection results demonstrate the effectiveness of MalDetConv over the state-of-the-art techniques with detection accuracy of 96.10%, 95.73%, 98.18%, and 99.93% achieved while detecting unseen malware from MalBehavD-V1, Allan and John, Brazilian, and Ki-D datasets, respectively. The experimental results show that MalDetConv is highly accurate in detecting both known and zero-day malware attacks on Windows devices

Intellectual Feature Ranking Model with Correlated Feature Set based Malware Detection in Cloud environment using Machine Learning

Malware detection for cloud systems has been studied extensively, and many different approaches have been developed and implemented in an effort to stay ahead of this ever-evolving threat. Malware refers to any programme or defect that is designed to duplicate itself or cause damage to the system's hardware or software. These attacks are designed specifically to cause harm to operational systems, but they are invisible to the human eye. One of the most exciting developments in data storage and service delivery today is cloud computing. There are significant benefits to be gained over more conventional protection methods by making use of this fast evolving technology to protect computer-based systems from cyber-related threats. Assets to be secured may reside in any networked computing environment, including but not limited to Cyber Physical Systems (CPS), critical systems, fixed and portable computers, mobile devices, and the Internet of Things (IoT). Malicious software or malware refers to any programme that intentionally compromises a computer system in order to compromise its security, privacy, or availability. A cloud-based intelligent behavior analysis model for malware detection system using feature set is proposed to identify the ever-increasing malware attacks. The suggested system begins by collecting malware samples from several virtual machines, from which unique characteristics can be extracted easily. Then, the malicious and safe samples are separated using the features provided to the learning-based and rule-based detection agents. To generate a relevant feature set for accurate malware detection, this research proposes an Intellectual Feature Ranking Model with Correlated Feature Set (IFR-CFS) model using enhanced logistic regression model for accurate detection of malware in the cloud environment. The proposed model when compared to the traditional feature selection model, performs better in generation of feature set for accurate detection of malware

Forensic identification and detection of hidden and obfuscated malware

The revolution in online criminal activities and malicious software (malware) has posed a serious challenge in malware forensics. Malicious attacks have become more organized and purposefully directed. With cybercrimes escalating to great heights in quantity as well as in sophistication and stealth, the main challenge is to detect hidden and obfuscated malware. Malware authors use a variety of obfuscation methods and specialized stealth techniques of information hiding to embed malicious code, to infect systems and to thwart any attempt to detect them, specifically with the use of commercially available anti-malware engines. This has led to the situation of zero-day attacks, where malware inflict systems even with existing security measures. The aim of this thesis is to address this situation by proposing a variety of novel digital forensic and data mining techniques to automatically detect hidden and obfuscated malware. Anti-malware engines use signature matching to detect malware where signatures are generated by human experts by disassembling the file and selecting pieces of unique code. Such signature based detection works effectively with known malware but performs poorly with hidden or unknown malware. Code obfuscation techniques, such as packers, polymorphism and metamorphism, are able to fool current detection techniques by modifying the parent code to produce offspring copies resulting in malware that has the same functionality, but with a different structure. These evasion techniques exploit the drawbacks of traditional malware detection methods, which take current malware structure and create a signature for detecting this malware in the future. However, obfuscation techniques aim to reduce vulnerability to any kind of static analysis to the determent of any reverse engineering process. Furthermore, malware can be hidden in file system slack space, inherent in NTFS file system based partitions, resulting in malware detection that even more difficult.Doctor of Philosoph

A Hybrid Multi-Filter Wrapper Feature Selection Method for Software Defect Predictors

Software Defect Prediction (SDP) is an approach used for identifying defect-prone software modules or components. It helps software engineer to optimally, allocate limited resources to defective software modules or components in the testing or maintenance phases of software development life cycle (SDLC). Nonetheless, the predictive performance of SDP models reckons largely on the quality of dataset utilized for training the predictive models. The high dimensionality of software metric features has been noted as a data quality problem which negatively affects the predictive performance of SDP models. Feature Selection (FS) is a well-known method for solving high dimensionality problem and can be divided into filter-based and wrapper-based methods. Filter-based FS has low computational cost, but the predictive performance of its classification algorithm on the filtered data cannot be guaranteed. On the contrary, wrapper-based FS have good predictive performance but with high computational cost and lack of generalizability. Therefore, this study proposes a hybrid multi-filter wrapper method for feature selection of relevant and irredundant features in software defect prediction. The proposed hybrid feature selection will be developed to take advantage of filter-filter and filter-wrapper relationships to give optimal feature subsets, reduce its evaluation cycle and subsequently improve SDP models overall predictive performance in terms of Accuracy, Precision and Recall values

CHID : conditional hybrid intrusion detection system for reducing false positives and resource consumption on malicous datasets

Inspecting packets to detect intrusions faces challenges when coping with a high volume of network traffic. Packet-based detection processes every payload on the wire, which degrades the performance of network intrusion detection system (NIDS). This issue requires an introduction of a flow-based NIDS that reduces the amount of data to be processed by examining aggregated information of related packets. However, flow-based detection still suffers from the generation of the false positive alerts due to incomplete data input. This study proposed a Conditional Hybrid Intrusion Detection (CHID) by combining the flow-based with packet-based detection. In addition, it is also aimed to improve the resource consumption of the packet-based detection approach. CHID applied attribute wrapper features evaluation algorithms that marked malicious flows for further analysis by the packet-based detection. Input Framework approach was employed for triggering packet flows between the packetbased and flow-based detections. A controlled testbed experiment was conducted to evaluate the performance of detection mechanism’s CHID using datasets obtained from on different traffic rates. The result of the evaluation showed that CHID gains a significant performance improvement in terms of resource consumption and packet drop rate, compared to the default packet-based detection implementation. At a 200 Mbps, CHID in IRC-bot scenario, can reduce 50.6% of memory usage and decreases 18.1% of the CPU utilization without packets drop. CHID approach can mitigate the false positive rate of flow-based detection and reduce the resource consumption of packet-based detection while preserving detection accuracy. CHID approach can be considered as generic system to be applied for monitoring of intrusion detection systems

A Weighted Minimum Redundancy Maximum Relevance Technique for Ransomware Early Detection in Industrial IoT

Ransomware attacks against Industrial Internet of Things (IIoT) have catastrophic consequences not only to the targeted infrastructure, but also the services provided to the public. By encrypting the operational data, the ransomware attacks can disrupt the normal operations, which represents a serious problem for industrial systems. Ransomware employs several avoidance techniques, such as packing, obfuscation, noise insertion, irrelevant and redundant system call injection, to deceive the security measures and make both static and dynamic analysis more difficult. In this paper, a Weighted minimum Redundancy maximum Relevance (WmRmR) technique was proposed for better feature significance estimation in the data captured during the early stages of ransomware attacks. The technique combines an enhanced mRMR (EmRmR) with the Term Frequency-Inverse Document Frequency (TF-IDF) so that it can filter out the runtime noisy behavior based on the weights calculated by the TF-IDF. The proposed technique has the capability to assess whether a feature in the relevant set is important or not. It has low-dimensional complexity and a smaller number of evaluations compared to the original mRmR method. The TF-IDF was used to evaluate the weights of the features generated by the EmRmR algorithm. Then, an inclusive entropy-based refinement method was used to decrease the size of the extracted data by identifying the system calls with strong behavioral indication. After extensive experimentation, the proposed technique has shown to be effective for ransomware early detection with low-complexity and few false-positive rates. To evaluate the proposed technique, we compared it with existing behavioral detection methods