Group-Wise Principal Component Analysis for Exploratory Intrusion Detection

Intrusion detection is a relevant layer of cybersecurity to prevent hacking and illegal activities from happening on the assets of corporations. Anomaly-based Intrusion Detection Systems perform an unsupervised analysis on data collected from the network and end systems, in order to identify singular events. While this approach may produce many false alarms, it is also capable of identifying new (zeroday) security threats. In this context, the use of multivariate approaches such as Principal Component Analysis (PCA) provided promising results in the past. PCA can be used in exploratory mode or in learning mode. Here, we propose an exploratory intrusion detection that replaces PCA with Group-wise PCA (GPCA), a recently proposed data analysis technique with additional exploratory characteristics. A main advantage of GPCA over PCA is that the former yields simple models, easy to understand by security professionals not trained in multivariate tools. Besides, the workflow in the intrusion detection with GPCA is more coherent with dominant strategies in intrusion detection. We illustrate the application of GPCA in two case studies.This work was supported in part by the Spanish Government-MINECO (Ministerio de Economía y Competitividad), using the Fondo Europeo de Desarrollo Regional (FEDER), under Projects TIN2014-60346-R and Project TIN2017-83494-R

Cross-product Penalized Component Analysis (XCAN)

Matrix factorization methods are extensively employed to understand complex data. In this paper, we introduce the cross-product penalized component analysis (XCAN), a sparse matrix factorization based on the optimization of a loss function that allows a trade-off between variance maximization and structural preservation. The approach is based on previous developments, notably (i) the Sparse Principal Component Analysis (SPCA) framework based on the LASSO, (ii) extensions of SPCA to constrain both modes of the factorization, like co-clustering or the Penalized Matrix Decomposition (PMD), and (iii) the Group-wise Principal Component Analysis (GPCA) method. The result is a flexible modeling approach that can be used for data exploration in a large variety of problems. We demonstrate its use with applications from different disciplines

A neural-visualization IDS for honeynet data

Neural intelligent systems can provide a visualization of the network traffic for security staff, in order to reduce the widely known high false-positive rate associated with misuse-based Intrusion Detection Systems (IDSs). Unlike previous work, this study proposes an unsupervised neural models that generate an intuitive visualization of the captured traffic, rather than network statistics. These snapshots of network events are immensely useful for security personnel that monitor network behavior. The system is based on the use of different neural projection and unsupervised methods for the visual inspection of honeypot data, and may be seen as a complementary network security tool that sheds light on internal data structures through visual inspection of the traffic itself. Furthermore, it is intended to facilitate verification and assessment of Snort performance (a well-known and widely-used misuse-based IDS), through the visualization of attack patterns. Empirical verification and comparison of the proposed projection methods are performed in a real domain, where two different case studies are defined and analyzedRegional Government of Gipuzkoa, the Department of Research, Education and Universities of the Basque Government, and the Spanish Ministry of Science and Innovation (MICINN) under projects TIN2010-21272-C02-01 and CIT-020000-2009-12 (funded by the European Regional Development Fund). This work was also supported in the framework of the IT4Innovations Centre of Excellence project, reg. no. CZ.1.05/1.1.00/02.0070 supported by the Operational Program 'Research and Development for Innovations' funded through the Structural Funds of the European Union and the state budget of the Czech RepublicElectronic version of an article published as International Journal of Neural Systems, Volume 22, Issue 02, April 2012 10.1142/S0129065712500050 ©copyright World Scientific Publishing Company http://www.worldscientific.com/worldscinet/ijn

Clustering extension of MOVICAB-IDS to distinguish intrusions in flow-based data

Much effort has been devoted to research on intrusion detection (ID) in recent years because intrusion strategies and technologies are constantly and quickly evolving. As an innovative solution based on visualization, MObile VIsualisation Connectionist Agent-Based IDS was previously proposed, conceived as a hybrid-intelligent ID System. It was designed to analyse continuous network data at a packet level and is extended in present paper for the analysis of flow-based traffic data. By incorporating clustering techniques to the original proposal, network flows are investigated trying to identify different types of attacks. The analysed real-life data (the well-known dataset from the University of Twente) come from a honeypot directly connected to the Internet (thus ensuring attack-exposure) and is analysed by means of clustering and neural techniques, individually and in conjunction. Promising results are obtained, proving the validity of the proposed extension for the analysis of network flow dat

Networkmetrics unraveled: MBDA in Action

We propose networkmetrics, a new data-driven approach for monitoring, troubleshooting and understanding communication networks using multivariate analysis. Networkmetric models are powerful machine-learning tools to interpret and interact with data collected from a network. In this paper, we illustrate the application of Multivariate Big Data Analysis (MBDA), a recently proposed networkmetric method with application to Big Data sets. We use MBDA for the detection and troubleshooting of network problems in a campus-wide Wi-Fi network. Data includes a seven-year trace (from 2012 to 2018) of the network’s most recent activity, with approximately 3,000 distinct access points, 40,000 authenticated users, and 600,000 distinct Wi-Fi stations. This is the longest and largest Wi-Fi trace known to date. To analyze this data, we propose learning and visualization procedures that extend MBDA. These procedures result in a methodology that allows network analysts to identify problems and diagnose and troubleshoot them, optimizing the network performance. In the paper, we go through the entire workflow of the approach, illustrating its application in detail and discussing processing times for parallel hardware

Networkmetrics unraveled: MBDA in Action

We propose networkmetrics, a new data-driven approach for monitoring, troubleshooting and understanding communication networks using multivariate analysis. Networkmetric models are powerful machine-learning tools to interpret and interact with data collected from a network. In this paper, we illustrate the application of Multivariate Big Data Analysis (MBDA), a recently proposed networkmetric method with application to Big Data sets. We use MBDA for the detection and troubleshooting of network problems in a campus-wide Wi-Fi network. Data includes a seven-year trace (from 2012 to 2018) of the network's most recent activity, with approximately 3,000 distinct access points, 40,000 authenticated users, and 600,000 distinct Wi-Fi stations. This is the longest and largest Wi-Fi trace known to date. To analyze this data, we propose learning and visualization procedures that extend MBDA. These procedures result in a methodology that allows network analysts to identify problems and diagnose and troubleshoot them, optimizing the network performance. In the paper, we go through the entire workflow of the approach, illustrating its application in detail and discussing processing times for parallel hardware

A FRAMEWORK FOR THE EVALUATION OF CYBERSECURITY EFFECTIVENESS OF ABU DHABI GOVERNMENT ENTITIES

Cyberspace has become one of the new frontiers for countries to demonstrate their power to survive in the digitized world. The UAE has become a major target for cyber conflicts due to the rapid increase in economic activity and technology. Further, the widespread use of the internet in the region to the tune of 88% by the end of 2014 has exposed the critical infrastructure to all forms of cyber threats. In this dissertation, the researcher presents a detailed study of the existing cybersecurity defences globally and an investigation into the factors that influence the effectiveness of cybersecurity defences in Abu Dhabi government entities. Further, the role of cybersecurity education, training, and awareness in enhancing the effectiveness of cybersecurity and the role of senior management in providing strategic direction to government entities on cybersecurity are evaluated in addition to determining the contribution of strategic planning and technology level in ensuring an effective cybersecurity system. The study has evaluated the level of Cybersecurity Effectiveness (CSE) in Abu Dhabi Government Entities and the results show that Science and Technology entity performed better than all other Entities with CSE Mean = 4.37 while Public Order showed the least performance with CSE Mean = 3.83 and the combined model of six factors with R-square value 0.317 after multiple regression implying that 32% change in CSE in the government entities is occurring due to the six (6) independent variables used in the study. Further, results show that management has the responsibility of putting in place strategies, frameworks and policies that respond appropriately to the prevention, detection and mitigation of cyberattacks. Results further indicate that culture-sensitive training and awareness programmes add to the quality and effectiveness of cybersecurity systems in government entities. Further, study findings reveal that qualified and experienced personnel in government entities show a greater understanding of cyber and information security issues. Finally, the researcher proposes a cybersecurity framework and a checklist, with checkpoints, for evaluating the effectiveness of cybersecurity systems within government entities and future research interventions

Practical Attacks Against Graph-based Clustering

Graph modeling allows numerous security problems to be tackled in a general way, however, little work has been done to understand their ability to withstand adversarial attacks. We design and evaluate two novel graph attacks against a state-of-the-art network-level, graph-based detection system. Our work highlights areas in adversarial machine learning that have not yet been addressed, specifically: graph-based clustering techniques, and a global feature space where realistic attackers without perfect knowledge must be accounted for (by the defenders) in order to be practical. Even though less informed attackers can evade graph clustering with low cost, we show that some practical defenses are possible.Comment: ACM CCS 201