Developing reliable anomaly detection system for critical hosts: a proactive defense paradigm

Current host-based anomaly detection systems have limited accuracy and incur high processing costs. This is due to the need for processing massive audit data of the critical host(s) while detecting complex zero-day attacks which can leave minor, stealthy and dispersed artefacts. In this research study, this observation is validated using existing datasets and state-of-the-art algorithms related to the construction of the features of a host's audit data, such as the popular semantic-based extraction and decision engines, including Support Vector Machines, Extreme Learning Machines and Hidden Markov Models. There is a challenging trade-off between achieving accuracy with a minimum processing cost and processing massive amounts of audit data that can include complex attacks. Also, there is a lack of a realistic experimental dataset that reflects the normal and abnormal activities of current real-world computers. This thesis investigates the development of new methodologies for host-based anomaly detection systems with the specific aims of improving accuracy at a minimum processing cost while considering challenges such as complex attacks which, in some cases, can only be visible via a quantified computing resource, for example, the execution times of programs, the processing of massive amounts of audit data, the unavailability of a realistic experimental dataset and the automatic minimization of the false positive rate while dealing with the dynamics of normal activities. This study provides three original and significant contributions to this field of research which represent a marked advance in its body of knowledge. The first major contribution is the generation and release of a realistic intrusion detection systems dataset as well as the development of a metric based on fuzzy qualitative modeling for embedding the possible quality of realism in a dataset's design process and assessing this quality in existing or future datasets. The second key contribution is constructing and evaluating the hidden host features to identify the trivial differences between the normal and abnormal artefacts of hosts' activities at a minimum processing cost. Linux-centric features include the frequencies and ranges, frequency-domain representations and Gaussian interpretations of system call identifiers with execution times while, for Windows, a count of the distinct core Dynamic Linked Library calls is identified as a hidden host feature. The final key contribution is the development of two new anomaly-based statistical decision engines for capitalizing on the potential of some of the suggested hidden features and reliably detecting anomalies. The first engine, which has a forensic module, is based on stochastic theories including Hierarchical hidden Markov models and the second is modeled using Gaussian Mixture Modeling and Correntropy. The results demonstrate that the proposed host features and engines are competent for meeting the identified challenges

Novel Estimation and Detection Techniques for 5G Networks

The thesis presents several detection and estimation techniques that can be incorporated into the fifth-generation (5G) networks. First, the thesis presents a novel system for orthogonal frequency division multiplexing (OFDM) to estimate the channel blindly. The system is based on modulating particular pairs of subcarriers using amplitude shift keying (ASK) and phase-shift keying (PSK) adjacent in the frequency domain, which enables the realization of a decision-directed (DD) one-shot blind channel estimator (OSBCE). The performance of the proposed estimator is evaluated in terms of the mean squared error (MSE), where an accurate analytical expression is derived and verified using Monte Carlo simulation under various channel conditions. The system has also extended to exploits the channel correlation over consecutive OFDM symbols to estimate the channel parameters blindly. Furthermore, a reliable and accurate approach has been introduced to evaluate the spectral efficiency of various communications systems. The metric takes into consideration the system dynamics, QoS requirements, and design constraints. Next, a novel efficient receiver design for wireless communication systems that incorporate OFDM transmission has been proposed. The proposed receiver does not require channel estimation or equalization to perform coherent data detection. Instead, channel estimation, equalization, and data detection are combined into a single operation, and hence, the detector performs a direct data detector (D3). The performance of the proposed D3 is thoroughly analyzed theoretically in terms of bit error rate (BER), where closed-form accurate approximations are derived for several cases of interest, and validated by Monte Carlo simulations. The computational complexity of D3 depends on the length of the sequence to be detected. Nevertheless, a significant complexity reduction can be achieved using the Viterbi algorithm (VA). Finally, the thesis proposes a low-complexity algorithm for detecting anomalies in industrial steelmaking furnaces operation. The algorithm utilizes the vibration measurements collected from several built-in sensors to compute the temporal correlation using the autocorrelation function (ACF). Furthermore, the proposed model parameters are tuned by solving multi-objective optimization using a genetic algorithm (GA). The proposed algorithm is tested using a practical dataset provided by an industrial steelmaking plant

CLADAG 2021 BOOK OF ABSTRACTS AND SHORT PAPERS

The book collects the short papers presented at the 13th Scientific Meeting of the Classification and Data Analysis Group (CLADAG) of the Italian Statistical Society (SIS). The meeting has been organized by the Department of Statistics, Computer Science and Applications of the University of Florence, under the auspices of the Italian Statistical Society and the International Federation of Classification Societies (IFCS). CLADAG is a member of the IFCS, a federation of national, regional, and linguistically-based classification societies. It is a non-profit, non-political scientific organization, whose aims are to further classification research

Detecting Anomalous Behavior in Cloud Servers by Nested-Arc Hidden SEMI-Markov Model with State Summarization

Anomaly detection for cloud servers is important for detecting zero-day attacks. However, it is very challenging due to the large amount of accumulated data. In this paper, a new mathematical model for modeling dynamic usage behavior and detecting anomalies is proposed. It is constructed using state summarization and a novel nested-arc hidden semi-Markov model (NAHSMM). State summarization is designed to extract usage behavior reflective states from a raw sequence. The NAHSMM is comprised of exterior and interior hidden Markov chains. The exterior controls the propagation of raw sequences of system calls and, conditional on it, the interior one controls the summarized observation process from the transition less usage behavior reflective states. An anomaly detection algorithm is derived by integrating state summarization and NAHSMM. During training the algorithm is assisted by a forensic module to tune the behavioral threshold. Experimental data is collected using IXIA Perfect Storm in conjunction with the commercial security-test hardware platform cyber range. To evaluate the reliability of the proposed model, first, its accuracy and training costs are compared with those of existing machine-learning models and then its scalability and resistance capabilities are tested. The results indicate that this model could be used as a method for detecting anomalies in cloud servers

WiFi-Based Human Activity Recognition Using Attention-Based BiLSTM

Recently, significant efforts have been made to explore human activity recognition (HAR) techniques that use information gathered by existing indoor wireless infrastructures through WiFi signals without demanding the monitored subject to carry a dedicated device. The key intuition is that different activities introduce different multi-paths in WiFi signals and generate different patterns in the time series of channel state information (CSI). In this paper, we propose and evaluate a full pipeline for a CSI-based human activity recognition framework for 12 activities in three different spatial environments using two deep learning models: ABiLSTM and CNN-ABiLSTM. Evaluation experiments have demonstrated that the proposed models outperform state-of-the-art models. Also, the experiments show that the proposed models can be applied to other environments with different configurations, albeit with some caveats. The proposed ABiLSTM model achieves an overall accuracy of 94.03%, 91.96%, and 92.59% across the 3 target environments. While the proposed CNN-ABiLSTM model reaches an accuracy of 98.54%, 94.25% and 95.09% across those same environments