Machine learning based anomaly detection for industry 4.0 systems.

223 p.This thesis studies anomaly detection in industrial systems using technologies from the Fourth Industrial Revolution (4IR), such as the Internet of Things, Artificial Intelligence, 3D Printing, and Augmented Reality. The goal is to provide tools that can be used in real-world scenarios to detect system anomalies, intending to improve production and maintenance processes. The thesis investigates the applicability and implementation of 4IR technology architectures, AI-driven machine learning systems, and advanced visualization tools to support decision-making based on the detection of anomalies. The work covers a range of topics, including the conception of a 4IR system based on a generic architecture, the design of a data acquisition system for analysis and modelling, the creation of ensemble supervised and semi-supervised models for anomaly detection, the detection of anomalies through frequency analysis, and the visualization of associated data using Visual Analytics. The results show that the proposed methodology for integrating anomaly detection systems in new or existing industries is valid and that combining 4IR architectures, ensemble machine learning models, and Visual Analytics tools significantly enhances theanomaly detection processes for industrial systems. Furthermore, the thesis presents a guiding framework for data engineers and end-users

산업에서 딥러닝을 이용한 시계열 데이터 분석

학위논문 (박사)-- 서울대학교 대학원 : 자연과학대학 협동과정 계산과학전공, 2019. 2. 강명주.딥러닝은 최근 몇 년간 산업 수학에 있어서 가장 강력하고 중요시 여겨지는 방법이다. 우리는 산업 수학의 시계열 데이터에 있어서 딥러닝 모델을 분석 및 예측 등에 정의하였다. 첫 번째로, 이상 감지를 위한 새로운 딥러닝 모델을 개발하였으며 이는 다양한 길이 뿐만 아니라 노이즈, 시간 차가 있는 데이터에서도 엔지니어에게 필요한 시계열 데이터 분석을 할 수 있었다. 두 번째로, 금융 시장의 트렌드를 예측하기 위한 다양한 딥러닝 모델을 개발 및 시험해보았으며 이 중 가중치 어텐션 네트워크의 경우 높은 예측 정확도 뿐만 아니라 시각화를 통해 직관적으로 모델을 이해 및 예측한 이유를 분석할 수 있었다.Deep learning, also called as artificial neural networks, is one of the most important and powerful subjects in industrial in recent years. Deep learning starts to show a great performance from image classification and in these days it have been applied to fields including computer vision, natural language process, speech recognition and etc. The performance is better than not only previous machine learning techniques, but also human experts in some cases. For an area with time series data, recurrent neural networks is widely used algorithm of deep learning. The aim of this theseis is to apply deep learning, especially with recurrent neural networks, for an industrial such as anomaly detection and trend prediction in financial market, with time series data . Its main contributions are (1) a new model for anomaly detection in time series data even for various length inputs, (2) various neural architectures for prediction in finance, and (3) attention networks and model analysis with attention vectors. Each experimental results of applications show better performances than previous machine learning techniques.1 Introduction 2 Deep Learning Background 2.1 Neural Networks 2.2 Various Activation Functions 2.3 Error Backpropagation 2.4 Regularization 2.4.1 Dropout 2.4.2 Batch Normalization 3 Deep Learning Models 3.1 Multi Layer Perceptron 3.2 Convolutional Neural Networks 3.3 Recurrent Neural Networks 3.4 Long Short Term Memory 3.5 Attention Networks 4 Anomaly Detection 4.1 Related Works of Anomaly Detection 4.1.1 Anomaly detection 4.1.2 t-SNE 4.1.3 Clustering 4.2 Deep Correlation Mapping 4.2.1 LSTM 4.2.2 t-SNE 4.2.3 Full Model Architecture 4.2.4 Anomaly detection using Deep Correlation Mapping 4.3 Experimental Results 4.3.1 Correlation 4.3.2 Anomaly detection using DeepCorr 4.4 Conclusion 5 Trend Prediction 5.1 Related works of Trend Prediction 5.2 Trend Prediction with Deep Learning Models 5.2.1 Dataset 5.2.2 MLP 5.2.3 1D-CNN 5.2.4 LSTM 5.2.5 Attention Networks 5.2.6 Weighted Attention Networks 5.3 Experimental Results 5.3.1 Best Lookback Days 5.3.2 Results of Various Deep Learning Models 5.3.3 Visualization Attention Vectors 5.4 Conclusion 6 Conclusion and Future Works Abstract (in Korean) Acknowledgement (in Korean)Docto

Sintel: A Machine Learning Framework to Extract Insights from Signals

The detection of anomalies in time series data is a critical task with many monitoring applications. Existing systems often fail to encompass an end-to-end detection process, to facilitate comparative analysis of various anomaly detection methods, or to incorporate human knowledge to refine output. This precludes current methods from being used in real-world settings by practitioners who are not ML experts. In this paper, we introduce Sintel, a machine learning framework for end-to-end time series tasks such as anomaly detection. The framework uses state-of-the-art approaches to support all steps of the anomaly detection process. Sintel logs the entire anomaly detection journey, providing detailed documentation of anomalies over time. It enables users to analyze signals, compare methods, and investigate anomalies through an interactive visualization tool, where they can annotate, modify, create, and remove events. Using these annotations, the framework leverages human knowledge to improve the anomaly detection pipeline. We demonstrate the usability, efficiency, and effectiveness of Sintel through a series of experiments on three public time series datasets, as well as one real-world use case involving spacecraft experts tasked with anomaly analysis tasks. Sintel's framework, code, and datasets are open-sourced at https://github.com/sintel-dev/.Comment: This work is accepted by ACM SIGMOD/PODS International Conference on Management of Data (SIGMOD 2022

Detecting Irregular Patterns in IoT Streaming Data for Fall Detection

Detecting patterns in real time streaming data has been an interesting and challenging data analytics problem. With the proliferation of a variety of sensor devices, real-time analytics of data from the Internet of Things (IoT) to learn regular and irregular patterns has become an important machine learning problem to enable predictive analytics for automated notification and decision support. In this work, we address the problem of learning an irregular human activity pattern, fall, from streaming IoT data from wearable sensors. We present a deep neural network model for detecting fall based on accelerometer data giving 98.75 percent accuracy using an online physical activity monitoring dataset called "MobiAct", which was published by Vavoulas et al. The initial model was developed using IBM Watson studio and then later transferred and deployed on IBM Cloud with the streaming analytics service supported by IBM Streams for monitoring real-time IoT data. We also present the systems architecture of the real-time fall detection framework that we intend to use with mbientlabs wearable health monitoring sensors for real time patient monitoring at retirement homes or rehabilitation clinics.Comment: 7 page

Data analytics for modeling and visualizing attack behaviors: A case study on SSH brute force attacks

In this research, we explore a data analytics based approach for modeling and visualizing attack behaviors. To this end, we employ Self-Organizing Map and Association Rule Mining algorithms to analyze and interpret the behaviors of SSH brute force attacks and SSH normal traffic as a case study. The experimental results based on four different data sets show that the patterns extracted and interpreted from the SSH brute force attack data sets are similar to each other but significantly different from those extracted from the SSH normal traffic data sets. The analysis of the attack traffic provides insight into behavior modeling for brute force SSH attacks. Furthermore, this sheds light into how data analytics could help in modeling and visualizing attack behaviors in general in terms of data acquisition and feature extraction