AI Solutions for MDS: Artificial Intelligence Techniques for Misuse Detection and Localisation in Telecommunication Environments

This report considers the application of Articial Intelligence (AI) techniques to the problem of misuse detection and misuse localisation within telecommunications environments. A broad survey of techniques is provided, that covers inter alia rule based systems, model-based systems, case based reasoning, pattern matching, clustering and feature extraction, articial neural networks, genetic algorithms, arti cial immune systems, agent based systems, data mining and a variety of hybrid approaches. The report then considers the central issue of event correlation, that is at the heart of many misuse detection and localisation systems. The notion of being able to infer misuse by the correlation of individual temporally distributed events within a multiple data stream environment is explored, and a range of techniques, covering model based approaches, `programmed' AI and machine learning paradigms. It is found that, in general, correlation is best achieved via rule based approaches, but that these suffer from a number of drawbacks, such as the difculty of developing and maintaining an appropriate knowledge base, and the lack of ability to generalise from known misuses to new unseen misuses. Two distinct approaches are evident. One attempts to encode knowledge of known misuses, typically within rules, and use this to screen events. This approach cannot generally detect misuses for which it has not been programmed, i.e. it is prone to issuing false negatives. The other attempts to `learn' the features of event patterns that constitute normal behaviour, and, by observing patterns that do not match expected behaviour, detect when a misuse has occurred. This approach is prone to issuing false positives, i.e. inferring misuse from innocent patterns of behaviour that the system was not trained to recognise. Contemporary approaches are seen to favour hybridisation, often combining detection or localisation mechanisms for both abnormal and normal behaviour, the former to capture known cases of misuse, the latter to capture unknown cases. In some systems, these mechanisms even work together to update each other to increase detection rates and lower false positive rates. It is concluded that hybridisation offers the most promising future direction, but that a rule or state based component is likely to remain, being the most natural approach to the correlation of complex events. The challenge, then, is to mitigate the weaknesses of canonical programmed systems such that learning, generalisation and adaptation are more readily facilitated

Cloud Computing Security, An Intrusion Detection System for Cloud Computing Systems

Cloud computing is widely considered as an attractive service model because it minimizes investment since its costs are in direct relation to usage and demand. However, the distributed nature of cloud computing environments, their massive resource aggregation, wide user access and efficient and automated sharing of resources enable intruders to exploit clouds for their advantage. To combat intruders, several security solutions for cloud environments adopt Intrusion Detection Systems. However, most IDS solutions are not suitable for cloud environments, because of problems such as single point of failure, centralized load, high false positive alarms, insufficient coverage for attacks, and inflexible design. The thesis defines a framework for a cloud based IDS to face the deficiencies of current IDS technology. This framework deals with threats that exploit vulnerabilities to attack the various service models of a cloud system. The framework integrates behaviour based and knowledge based techniques to detect masquerade, host, and network attacks and provides efficient deployments to detect DDoS attacks. This thesis has three main contributions. The first is a Cloud Intrusion Detection Dataset (CIDD) to train and test an IDS. The second is the Data-Driven Semi-Global Alignment, DDSGA, approach and three behavior based strategies to detect masquerades in cloud systems. The third and final contribution is signature based detection. We introduce two deployments, a distributed and a centralized one to detect host, network, and DDoS attacks. Furthermore, we discuss the integration and correlation of alerts from any component to build a summarized attack report. The thesis describes in details and experimentally evaluates the proposed IDS and alternative deployments. Acknowledgment: =============== • This PH.D. is achieved through an international joint program with a collaboration between University of Pisa in Italy (Department of Computer Science, Galileo Galilei PH.D. School) and University of Arizona in USA (College of Electrical and Computer Engineering). • The PHD topic is categorized in both Computer Engineering and Information Engineering topics. • The thesis author is also known as "Hisham A. Kholidy"

Towards Efficient Intrusion Detection using Hybrid Data Mining Techniques

The enormous development in the connectivity among different type of networks poses significant concerns in terms of privacy and security. As such, the exponential expansion in the deployment of cloud technology has produced a massive amount of data from a variety of applications, resources and platforms. In turn, the rapid rate and volume of data creation in high-dimension has begun to pose significant challenges for data management and security. Handling redundant and irrelevant features in high-dimensional space has caused a long-term challenge for network anomaly detection. Eliminating such features with spectral information not only speeds up the classification process, but also helps classifiers make accurate decisions during attack recognition time, especially when coping with large-scale and heterogeneous data such as network traffic data. Furthermore, the continued evolution of network attack patterns has resulted in the emergence of zero-day cyber attacks, which nowadays has considered as a major challenge in cyber security. In this threat environment, traditional security protections like firewalls, anti-virus software, and virtual private networks are not always sufficient. With this in mind, most of the current intrusion detection systems (IDSs) are either signature-based, which has been proven to be insufficient in identifying novel attacks, or developed based on absolute datasets. Hence, a robust mechanism for detecting intrusions, i.e. anomaly-based IDS, in the big data setting has therefore become a topic of importance. In this dissertation, an empirical study has been conducted at the initial stage to identify the challenges and limitations in the current IDSs, providing a systematic treatment of methodologies and techniques. Next, a comprehensive IDS framework has been proposed to overcome the aforementioned shortcomings. First, a novel hybrid dimensionality reduction technique is proposed combining information gain (IG) and principal component analysis (PCA) methods with an ensemble classifier based on three different classification techniques, named IG-PCA-Ensemble. Experimental results show that the proposed dimensionality reduction method contributes more critical features and reduced the detection time significantly. The results show that the proposed IG-PCA-Ensemble approach has also exhibits better performance than the majority of the existing state-of-the-art approaches

An Artificial Neural Network-based Decision-Support System for Integrated Network Security

As large-scale Cyber attacks become more sophisticated, local network defenders should employ strength-in-numbers to achieve mission success. Group collaboration reduces individual efforts to analyze and assess network traffic. Network defenders must evolve from an isolated defense in sector policy and move toward a collaborative strength-in-numbers defense policy that rethinks traditional network boundaries. Such a policy incorporates a network watch ap-proach to global threat defense, where local defenders share the occurrence of local threats in real-time across network security boundaries, increases Cyber Situation Awareness (CSA) and provides localized decision-support. A single layer feed forward artificial neural network (ANN) is employed as a global threat event recommender system (GTERS) that learns expert-based threat mitigation decisions. The system combines the occurrence of local threat events into a unified global event situation, forming a global policy that allows the flexibility of various local policy interpretations of the global event. Such flexibility enables a Linux based network defender to ignore windows-specific threats while focusing on Linux threats in real-time. In this thesis, the GTERS is shown to effectively encode an arbitrary policy with 99.7% accuracy based on five threat-severity levels and achieves a generalization accuracy of 96.35% using four distinct participants and 9-fold cross-validation

Intrusion detection system for IoT networks for detection of DDoS attacks

PhD ThesisIn this thesis, a novel Intrusion Detection System (IDS) based on the hybridization of the Deep Learning (DL) technique and the Multi-objective Optimization method for the detection of Distributed Denial of Service (DDoS) attacks in Internet of Things (IoT) networks is proposed. IoT networks consist of different devices with unique hardware and software configurations communicating over different communication protocols, which produce huge multidimensional data that make IoT networks susceptible to cyber-attacks. The network IDS is a vital tool for protecting networks against threats and malicious attacks. Existing systems face significant challenges due to the continuous emergence of new and more sophisticated cyber threats that are not recognized by them, and therefore advanced IDS is required. This thesis focusses especially on the DDoS attack that is one of the cyber-attacks that has affected many IoT networks in recent times and had resulted in substantial devastating losses. A thorough literature review is conducted on DDoS attacks in the context of IoT networks, IDSs available especially for the IoT networks and the scope and applicability of DL methodology for the detection of cyber-attacks. This thesis includes three main contributions for 1) developing a feature selection algorithm for an IoT network fulfilling six important objectives, 2) designing four DL models for the detection of DDoS attacks and 3) proposing a novel IDS for IoT networks. In the proposed work, for developing advanced IDS, a Jumping Gene adapted NSGA-II multi-objective optimization algorithm for reducing the dimensionality of massive IoT data and Deep Learning model consisting of a Convolutional Neural Network (CNN) combined with Long Short-Term Memory (LSTM) for classification are employed. The experimentation is conducted using a High-Performance Computer (HPC) on the latest CISIDS2017 datasets for DDoS attacks and achieved an accuracy of 99.03 % with a 5-fold reduction in training time. The proposed method is compared with machine learning (ML) algorithms and other state-of-the-art methods, which confirms that the proposed method outperforms other approaches.Government of Indi

Android Malware Detection System using Genetic Programming

Nowadays, smartphones and other mobile devices are playing a significant role in the way people engage in entertainment, communicate, network, work, and bank and shop online. As the number of mobile phones sold has increased dramatically worldwide, so have the security risks faced by the users, to a degree most do not realise. One of the risks is the threat from mobile malware. In this research, we investigate how supervised learning with evolutionary computation can be used to synthesise a system to detect Android mobile phone attacks. The attacks include malware, ransomware and mobile botnets. The datasets used in this research are publicly downloadable, available for use with appropriate acknowledgement. The primary source is Drebin. We also used ransomware and mobile botnet datasets from other Android mobile phone researchers. The research in this thesis uses Genetic Programming (GP) to evolve programs to distinguish malicious and non-malicious applications in Android mobile datasets. It also demonstrates the use of GP and Multi-Objective Evolutionary Algorithms (MOEAs) together to explore functional (detection rate) and non-functional (execution time and power consumption) trade-offs. Our results show that malicious and non-malicious applications can be distinguished effectively using only the permissions held by applications recorded in the application's Android Package (APK). Such a minimalist source of features can serve as the basis for highly efficient Android malware detection. Non-functional tradeoffs are also highlight

Transparent, trustworthy and privacy-preserving supply chains

Over the years, supply chains have evolved from a few regional traders to globally complex chains of trade. Consequently, supply chain management systems have become heavily dependent on digitization for the purpose of data storage and traceability of goods. However, these traceability systems suffer from issues such as scattering of information across multiple silos and susceptibility of erroneous or modified data and thus are often unable to provide reliable information about a product. Due to propriety reasons, often end-to-end traceability is not available to the general consumer. The second issue is ensuring the credibility of the collated information about a product. The digital data may not be the true representation of the physical events which raises the issues of trusting the available information. If the source of digital data is not trustworthy, the provenance or traceability of a product becomes questionable. The third issue in supply chain management is a trade-off between the provenance information and protection of this data. The information is often associated with the identity of the contributing entity to ensure trust. However, the identity association makes it difficult to protect trade secrets such as shipments, pricing, and trade frequency of traders while simultaneously ensuring the provenance/traceability to the consumers. Our work aims to address above mentioned challenges related to traceability, trustworthiness and privacy. To support traceability and provenance, a consortium blockchain based framework, ProductChain, is proposed which provides an immutable audit trail of the supply chain events pertaining to the product and its origin. The framework also presents a sharded network model to meet the scalability needs of complex supply chains. Simulation results for our Proof of Concept (PoC) implementation show that query time for retrieving end-to-end traceability is of the order of a few milliseconds even when the information is collated from multiple regional blockchains. Next, to ensure the credibility of data from the supply chain entities, it is important to have an accountability mechanism which can penalise or reward the entities for their dishonest or honest contributions, respectively. We propose the TrustChain framework, which calculates a trust score for data contributing entities to the blockchain using multiple observations. These observations include feedback from interactions among supply chain entities, inputs from third party regulators and readings from IoT sensors. The integrated reputation system with blockchain, dynamically assigns trust and reputation scores to commodities and traders using smart contracts. A PoC implementation over Hyperledger Fabric shows that TrustChain incurs minimal overheads over a baseline. For protecting trade secrets while simultaneously ensuring traceability, PrivChain is proposed. PrivChain's framework allows traders to share computation or proofs in support of provenance and traceability claims rather than sharing the data itself. The framework also proposes an integrated incentive mechanism for traders providing such proofs. A PoC implementation on Hyperledger Fabric reveals a minimal overhead of using PrivChain as the data related computations are carried off-chain. Finally, we propose TradeChain which addresses the issue of preserving the privacy of identity related information with the blockchain data and gives greater access control to the data owners, i.e. traders. This framework decouples the identities of traders by managing two ledgers: one for managing decentralised identities and another for recording supply chain events. The information from both ledgers is then collated using access tokens provided by the data owners. In this way, they can dynamically control access to the blockchain data at a granular level. A PoC implementation is developed both on Hyperledger Indy and Fabric and we demonstrate minimal overheads for the different components of TradeChain

Applied Metaheuristic Computing

For decades, Applied Metaheuristic Computing (AMC) has been a prevailing optimization technique for tackling perplexing engineering and business problems, such as scheduling, routing, ordering, bin packing, assignment, facility layout planning, among others. This is partly because the classic exact methods are constrained with prior assumptions, and partly due to the heuristics being problem-dependent and lacking generalization. AMC, on the contrary, guides the course of low-level heuristics to search beyond the local optimality, which impairs the capability of traditional computation methods. This topic series has collected quality papers proposing cutting-edge methodology and innovative applications which drive the advances of AMC