Robustness and Generalization Performance of Deep Learning Models on Cyber-Physical Systems: A Comparative Study

Deep learning (DL) models have seen increased attention for time series forecasting, yet the application on cyber-physical systems (CPS) is hindered by the lacking robustness of these methods. Thus, this study evaluates the robustness and generalization performance of DL architectures on multivariate time series data from CPS. Our investigation focuses on the models' ability to handle a range of perturbations, such as sensor faults and noise, and assesses their impact on overall performance. Furthermore, we test the generalization and transfer learning capabilities of these models by exposing them to out-of-distribution (OOD) samples. These include deviations from standard system operations, while the core dynamics of the underlying physical system are preserved. Additionally, we test how well the models respond to several data augmentation techniques, including added noise and time warping. Our experimental framework utilizes a simulated three-tank system, proposed as a novel benchmark for evaluating the robustness and generalization performance of DL algorithms in CPS data contexts. The findings reveal that certain DL model architectures and training techniques exhibit superior effectiveness in handling OOD samples and various perturbations. These insights have significant implications for the development of DL models that deliver reliable and robust performance in real-world CPS applications.Comment: Accepted at the IJCAI 2023 Workshop of Artificial Intelligence for Time Series Analysis (AI4TS

Diagnosis driven Anomaly Detection for CPS

In Cyber-Physical Systems (CPS) research, anomaly detection (detecting abnormal behavior) and diagnosis (identifying the underlying root cause) are often treated as distinct, isolated tasks. However, diagnosis algorithms require symptoms, i.e. temporally and spatially isolated anomalies, as input. Thus, anomaly detection and diagnosis must be developed together to provide a holistic solution for diagnosis in CPS. We therefore propose a method for utilizing deep learning-based anomaly detection to generate inputs for Consistency-Based Diagnosis (CBD). We evaluate our approach on a simulated and a real-world CPS dataset, where our model demonstrates strong performance relative to other state-of-the-art models

AI-assisted study of auxetic structures

In this study, the viability of using machine learning models to predict stress-strain curves of auxetic structures based on geometry-describing parameters is explored. Given the computational cost and time associated with generating these curves through numerical simulations, a machine learning-based approach promises a more efficient alternative. A range of machine learning models, including Artificial Neural Networks, k-Nearest Neighbors Regression, Support Vector Regression, and XGBoost, is implemented and compared regarding the aptitude to predict stress-strain curves under quasi-static compressive loading. Training data is generated using validated finite element simulations. The performance of these models is rigorously tested on data not seen during training. The Feed-Forward Artificial Neural Network emerged as the most proficient model, achieving a Mean Absolute Percentage Error of 0.367 ± 0.230

AI-assisted study of auxetic structures

In this study, the viability of using machine learning models to predict stress-strain curves of auxetic structures based on geometry-describing parameters is explored. Given the computational cost and time associated with generating these curves through numerical simulations, a machine learning-based approach promises a more efficient alternative. A range of machine learning models, including Artificial Neural Networks, k-Nearest Neighbors Regression, Support Vector Regression, and XGBoost, is implemented and compared regarding the aptitude to predict stress-strain curves under quasi-static compressive loading. Training data is generated using validated finite element simulations. The performance of these models is rigorously tested on data not seen during training. The Feed-Forward Artificial Neural Network emerged as the most proficient model, achieving a Mean Absolute Percentage Error of 0.367 ± 0.230