The optimal input-independent baseline for binary classification: The Dutch Draw

Before any binary classification model is taken into practice, it is important to validate its performance on a proper test set. Without a frame of reference given by a baseline method, it is impossible to determine if a score is “good” or “bad.” The goal of this paper is to examine all baseline methods that are independent of feature values and determine which model is the “best” and why. By identifying which baseline models are optimal, a crucial selection decision in the evaluation process is simplified. We prove that the recently proposed Dutch Draw baseline is the best input-independent classifier (independent of feature values) for all order-invariant measures (independent of sequence order) assuming that the samples are randomly shuffled. This means that the Dutch Draw baseline is the optimal baseline under these intuitive requirements and should therefore be used in practice

Machine Learning Based Detection of False Data Injection Attacks in Wide Area Monitoring Systems

The Smart Grid (SG) is an upgraded, intelligent, and a more reliable version of the traditional Power Grid due to the integration of information and communication technologies. The operation of the SG requires a dense communication network to link all its components. But such a network renders it prone to cyber attacks jeopardizing the integrity and security of the communicated data between the physical electric grid and the control centers. One of the most prominent components of the SG are Wide Area Monitoring Systems (WAMS). WAMS are a modern platform for grid-wide information, communication, and coordination that play a major role in maintaining the stability of the grid against major disturbances. In this thesis, an anomaly detection framework is proposed to identify False Data Injection (FDI) attacks in WAMS using different Machine Learning (ML) and Deep Learning (DL) techniques, i.e., Deep Autoencoders (DAE), Long-Short Term Memory (LSTM), and One-Class Support Vector Machine (OC-SVM). These algorithms leverage diverse, complex, and high-volume power measurements coming from communications between different components of the grid to detect intelligent FDI attacks. The injected false data is assumed to target several major WAMS monitoring applications, such as Voltage Stability Monitoring (VSM), and Phase Angle Monitoring (PAM). The attack vector is considered to be smartly crafted based on the power system data, so that it can pass the conventional bad data detection schemes and remain stealthy. Due to the lack of realistic attack data, machine learning-based anomaly detection techniques are used to detect FDI attacks. To demonstrate the impact of attacks on the realistic WAMS traffic and to show the effectiveness of the proposed detection framework, a Hardware-In-the-Loop (HIL) co-simulation testbed is developed. The performance of the implemented techniques is compared on the testbed data using different metrics: Accuracy, F1 score, and False Positive Rate (FPR) and False Negative Rate (FNR). The IEEE 9-bus and IEEE 39-bus systems are used as benchmarks to investigate the framework scalability. The experimental results prove the effectiveness of the proposed models in detecting FDI attacks in WAMS

Survey of Network Intrusion Detection Methods from the Perspective of the Knowledge Discovery in Databases Process

The identification of cyberattacks which target information and communication systems has been a focus of the research community for years. Network intrusion detection is a complex problem which presents a diverse number of challenges. Many attacks currently remain undetected, while newer ones emerge due to the proliferation of connected devices and the evolution of communication technology. In this survey, we review the methods that have been applied to network data with the purpose of developing an intrusion detector, but contrary to previous reviews in the area, we analyze them from the perspective of the Knowledge Discovery in Databases (KDD) process. As such, we discuss the techniques used for the capture, preparation and transformation of the data, as well as, the data mining and evaluation methods. In addition, we also present the characteristics and motivations behind the use of each of these techniques and propose more adequate and up-to-date taxonomies and definitions for intrusion detectors based on the terminology used in the area of data mining and KDD. Special importance is given to the evaluation procedures followed to assess the different detectors, discussing their applicability in current real networks. Finally, as a result of this literature review, we investigate some open issues which will need to be considered for further research in the area of network security