A Novel Unsupervised Graph Wavelet Autoencoder for Mechanical System Fault Detection

Reliable fault detection is an essential requirement for safe and efficient operation of complex mechanical systems in various industrial applications. Despite the abundance of existing approaches and the maturity of the fault detection research field, the interdependencies between condition monitoring data have often been overlooked. Recently, graph neural networks have been proposed as a solution for learning the interdependencies among data, and the graph autoencoder (GAE) architecture, similar to standard autoencoders, has gained widespread use in fault detection. However, both the GAE and the graph variational autoencoder (GVAE) have fixed receptive fields, limiting their ability to extract multiscale features and model performance. To overcome these limitations, we propose two graph neural network models: the graph wavelet autoencoder (GWAE), and the graph wavelet variational autoencoder (GWVAE). GWAE consists mainly of the spectral graph wavelet convolutional (SGWConv) encoder and a feature decoder, while GWVAE is the variational form of GWAE. The developed SGWConv is built upon the spectral graph wavelet transform which can realize multiscale feature extraction by decomposing the graph signal into one scaling function coefficient and several spectral graph wavelet coefficients. To achieve unsupervised mechanical system fault detection, we transform the collected system signals into PathGraph by considering the neighboring relationships of each data sample. Fault detection is then achieved by evaluating the reconstruction errors of normal and abnormal samples. We carried out experiments on two condition monitoring datasets collected from fuel control systems and one acoustic monitoring dataset from a valve. The results show that the proposed methods improve the performance by around 3%~4% compared to the comparison methods

지도 및 비지도 학습을 이용한 로봇 머니퓰레이터 충돌 감지

학위논문(박사) -- 서울대학교대학원 : 공과대학 기계항공공학부, 2022.2. 박종우.사람과 공유된 구조화되지 않은 동적 환경에서 작동하는 협업 로봇 머니퓰레이터는 날카로운 충돌(경성 충돌)에서 더 긴 지속 시간의 밀고 당기는 동작(연성 충돌)에 이르기까지의 다양한 충돌을 빠르고 정확하게 감지해야 한다. 모터 전류 측정값을 이용해 외부 조인트 토크를 추정하는 동역학 모델 기반 감지 방법을 사용할 경우, 정확한 마찰 파라미터 모델링 및 식별과 같은 모터 마찰에 대한 적절한 처리가 필요하다. 올바르게 적용하면 매우 효과적이지만, 동역학과 마찰 파라미터를 모델링 및 식별하고 여러 개의 감지 임계값을 수동으로 설정하는 데에는 상당한 노력이 필요하기 때문에 대량 생산되는 산업용 로봇에 이를 적용하기는 어렵다. 또한 적절한 식별 후에도 동역학에 백래시, 탄성 등 모델링되지 않은 효과나 불확실성이 여전히 존재할 수 있다. 본 논문에서는 순수 모델 기반 방법의 구현 어려움을 피하고 불확실한 동역학적 효과를 보상하는 수단으로 로봇 머니퓰레이터를 위한 총 네 가지의 학습 기반 충돌 감지 방법을 제안한다. 두 개의 방법은 학습을 위해 충돌 및 비충돌 동작 데이터가 모두 필요한 지도 학습 알고리즘(서포트 벡터 머신 회귀, 일차원 합성곱 신경망 기반)을 사용하며 나머지 두 개의 방법은 학습을 위해 비충돌 동작 데이터만을 필요로 하는 비지도 이상치 탐지 알고리즘(단일 클래스 서포트 벡터 머신, 오토인코더 기반)에 기반한다. 로봇 동역학 모델과 모터 전류 측정값만을 필요로 하며 추가적인 외부 센서나 마찰 모델링, 여러 개의 감지 임계값에 대한 수동 조정은 필요하지 않다. 먼저 지도 및 비지도 감지 방법을 학습시키고 검증하는 데 사용되는, 6자유도 협업 로봇 머니퓰레이터를 이용해 수집된 로봇 충돌 데이터를 설명한다. 우리가 고려하는 충돌 시나리오는 경성 충돌, 연성 충돌, 비충돌 동작으로, 경성 및 연성 충돌은 모두 동일하게 충돌로 간주한다. 감지 성능 검증을 위한 테스트 데이터는 총 787건의 충돌과 62.4분의 비충돌 동작으로 이루어져 있으며, 이는 로봇이 랜덤 점대점 6관절 동작을 수행하는 동안 수집된다. 데이터 수집 중 로봇의 끝단에는 미부착, 3.3 kg, 5.0 kg의 세 가지 유형의 페이로드를 부착한다. 다음으로, 수집된 테스트 데이터를 이용해 지도 감지 방법의 감지 성능을 실험적으로 검증한다. 실험 결과는 지도 감지 방법이 가벼운 네트워크를 이용해 광범위한 경성 및 연성 충돌을 실시간으로 정확하게 감지할 수 있음을 보여주며, 이를 통해 모델 파라미터의 불확실성과 측정 노이즈, 백래시, 변형 등 모델링되지 않은 효과까지 보상됨을 알 수 있다. 또한 서포트 벡터 머신 회귀 기반 방법은 하나의 감지 임계값에 대한 조정만 필요하며 일차원 합성곱 신경망 기반 방법은 하나의 아웃풋 필터 파라미터에 대한 조정만 필요한데, 두 방법 모두 직관적인 감도 조정이 가능하다. 나아가 일련의 시뮬레이션 실험을 통해 지도 감지 방법의 일반화 성능을 실험적으로 검증한다. 마지막으로, 동일한 테스트 데이터에 대해 비지도 감지 방법의 감지 성능과 일반화 성능 또한 검증한다. 실험 결과는 비지도 감지 방법 또한 가벼운 계산과 하나의 감지 임계값에 대한 조정만으로 다양한 경성 및 연성 충돌을 실시간으로 강인하게 감지할 수 있음을 보여주며, 이를 통해 모델링되지 않은 마찰을 포함한 불확실한 동역학적 효과를 비지도 학습으로도 보상할 수 있음을 알 수 있다. 지도 감지 방법이 더 나은 감지 성능을 보이지만, 비지도 감지 방법은 학습을 위해 비충돌 동작 데이터만을 필요로 하며 발생할 수 있는 모든 유형의 충돌에 대한 정보를 필요로 하지 않기 때문에 대량 생산되는 산업용 로봇에 더 적합하다.Collaborative robot manipulators operating in dynamic and unstructured environments shared with humans require fast and accurate detection of collisions, which can range from sharp impacts (hard collisions) to pulling and pushing motions of longer duration (soft collisions). When using dynamics model-based detection methods that estimate the external joint torque with motor current measurements, proper treatment for friction in the motors is required, such as accurate modeling and identification of friction parameters. Although highly effective when done correctly, modeling and identifying the dynamics and friction parameters, and manually setting multiple detection thresholds require considerable effort, making them difficult to be replicated for mass-produced industrial robots. There may also still exist unmodeled effects or uncertainties in the dynamics even after proper identification, e.g., backlash, elasticity. This dissertation presents a total of four learning-based collision detection methods for robot manipulators as a means of sidestepping some of the implementation difficulties of pure model-based methods and compensating for uncertain dynamic effects. Two methods use supervised learning algorithms – support vector machine regression and a one-dimensional convolutional neural network-based – that require both the collision and collision-free motion data for training. The other two methods are based on unsupervised anomaly detection algorithms – a one-class support vector machine and an autoencoder-based – that require only the collision-free motion data for training. Only the motor current measurements together with a robot dynamics model are required while no additional external sensors, friction modeling, or manual tuning of multiple detection thresholds are needed. We first describe the robot collision dataset collected with a six-dof collaborative robot manipulator, which is used for training and validating our supervised and unsupervised detection methods. The collision scenarios we consider are hard collisions, soft collisions, and collision-free, where both hard and soft collisions are treated in the same manner as just collisions. The test dataset for detection performance verification includes a total of 787 collisions and 62.4 minutes of collision-free motions, all collected while the robot is executing random point-to-point six-joint motions. During data collection, three types of payloads are attached to the end-effector: no payload, 3.3 kg payload, and 5.0 kg payload. Then the detection performance of our supervised detection methods is experimentally verified with the collected test dataset. Results demonstrate that our supervised detection methods can accurately detect a wide range of hard and soft collisions in real-time using a light network, compensating for uncertainties in the model parameters as well as unmodeled effects like friction, measurement noise, backlash, and deformations. Moreover, the SVMR-based method requires only one constant detection threshold to be tuned while the 1-D CNN-based method requires only one output filter parameter to be tuned, both of which allow intuitive sensitivity tuning. Furthermore, the generalization capability of our supervised detection methods is experimentally verified with a set of simulation experiments. Finally, our unsupervised detection methods are also validated for the same test dataset; the detection performance and the generalization capability are verified. The experimental results show that our unsupervised detection methods are also able to robustly detect a variety of hard and soft collisions in real-time with very light computation and with only one constant detection threshold required to be tuned, validating that uncertain dynamic effects including the unmodeled friction can be successfully compensated also with unsupervised learning. Although our supervised detection methods show better detection performance, our unsupervised detection methods are more practical for mass-produced industrial robots since they require only the data for collision-free motions for training, and the knowledge of every possible type of collision that can occur is not required.1 Introduction 1 1.1 Model-Free Methods 2 1.2 Model-Based Methods 2 1.3 Learning-Based Methods 4 1.3.1 Using Supervised Learning Algorithms 5 1.3.2 Using Unsupervised Learning Algorithms 6 1.4 Contributions of This Dissertation 7 1.4.1 Supervised Learning-Based Model-Compensating Detection 7 1.4.2 Unsupervised Learning-Based Model-Compensating Detection 8 1.4.3 Comparison with Existing Detection Methods 9 1.5 Organization of This Dissertation 14 2 Preliminaries 17 2.1 Introduction 17 2.2 Robot Dynamics 17 2.3 Momentum Observer-Based Collision Detection 19 2.4 Supervised Learning Algorithms 21 2.4.1 Support Vector Machine Regression 21 2.4.2 One-Dimensional Convolutional Neural Network 23 2.5 Unsupervised Anomaly Detection 25 2.6 One-Class Support Vector Machine 26 2.7 Autoencoder-Based Anomaly Detection 28 2.7.1 Autoencoder Network Architecture and Training 28 2.7.2 Anomaly Detection Using Autoencoders 29 3 Robot Collision Data 31 3.1 Introduction 31 3.2 True Collision Index Labeling 31 3.3 Collision Scenarios 35 3.4 Monitoring Signal 36 3.5 Signal Normalization and Sampling 37 3.6 Test Data for Detection Performance Verification 39 4 Supervised Learning-Based Model-Compensating Detection 43 4.1 Introduction 43 4.2 SVMR-Based Collision Detection 44 4.2.1 Input Feature Vector Design 44 4.2.2 SVMR Training 45 4.2.3 Collision Detection Sensitivity Adjustment 46 4.3 1-D CNN-Based Collision Detection 50 4.3.1 Network Input Design 50 4.3.2 Network Architecture and Training 50 4.3.3 An Output Filtering Method to Reduce False Alarms 53 4.4 Collision Detection Performance Criteria 54 4.4.1 Area Under the Precision-Recall Curve (PRAUC) 54 4.4.2 Detection Delay and Number of Detection Failures 54 4.5 Collision Detection Performance Analysis 56 4.5.1 Global Performance with Varying Thresholds 56 4.5.2 Detection Delay and Number of Detection Failures 57 4.5.3 Real-Time Inference 60 4.6 Generalization Capability Analysis 60 4.6.1 Generalization to Small Perturbations 60 4.6.2 Generalization to an Unseen Payload 62 5 Unsupervised Learning-Based Model-Compensating Detection 67 5.1 Introduction 67 5.2 OC-SVM-Based Collision Detection 68 5.2.1 Input Feature Vector 68 5.2.2 OC-SVM Training 70 5.2.3 Collision Detection with the Trained OC-SVM 70 5.3 Autoencoder-Based Collision Detection 70 5.3.1 Network Input and Output 71 5.3.2 Network Architecture and Training 71 5.3.3 Collision Detection with the Trained Autoencoder 72 5.4 Collision Detection Performance Analysis 74 5.4.1 Global Performance with Varying Thresholds 75 5.4.2 Detection Delay and Number of Detection Failures 75 5.4.3 Comparison with Supervised Learning-Based Methods 80 5.4.4 Real-Time Inference 83 5.5 Generalization Capability Analysis 83 5.5.1 Generalization to Small Perturbations 84 5.5.2 Generalization to an Unseen Payload 85 6 Conclusion 89 6.1 Summary and Discussion 89 6.2 Future Work 93 A Appendix 95 A.1 SVM-Based Classification of Detected Collisions 95 A.2 Direct Estimation-Based Detection Methods 97 A.3 Model-Independent Supervised Detection Methods 101 A.4 Generalization to Large Changes in the Dynamics Model 102 Bibliography 106 Abstract 112박

IoT Anomaly Detection Methods and Applications: A Survey

Ongoing research on anomaly detection for the Internet of Things (IoT) is a rapidly expanding field. This growth necessitates an examination of application trends and current gaps. The vast majority of those publications are in areas such as network and infrastructure security, sensor monitoring, smart home, and smart city applications and are extending into even more sectors. Recent advancements in the field have increased the necessity to study the many IoT anomaly detection applications. This paper begins with a summary of the detection methods and applications, accompanied by a discussion of the categorization of IoT anomaly detection algorithms. We then discuss the current publications to identify distinct application domains, examining papers chosen based on our search criteria. The survey considers 64 papers among recent publications published between January 2019 and July 2021. In recent publications, we observed a shortage of IoT anomaly detection methodologies, for example, when dealing with the integration of systems with various sensors, data and concept drifts, and data augmentation where there is a shortage of Ground Truth data. Finally, we discuss the present such challenges and offer new perspectives where further research is required.Comment: 22 page

Autoencoder Based Iterative Modeling and Multivariate Time-Series Subsequence Clustering Algorithm

This paper introduces an algorithm for the detection of change-points and the identification of the corresponding subsequences in transient multivariate time-series data (MTSD). The analysis of such data has become more and more important due to the increase of availability in many industrial fields. Labeling, sorting or filtering highly transient measurement data for training condition based maintenance (CbM) models is cumbersome and error-prone. For some applications it can be sufficient to filter measurements by simple thresholds or finding change-points based on changes in mean value and variation. But a robust diagnosis of a component within a component group for example, which has a complex non-linear correlation between multiple sensor values, a simple approach would not be feasible. No meaningful and coherent measurement data which could be used for training a CbM model would emerge. Therefore, we introduce an algorithm which uses a recurrent neural network (RNN) based Autoencoder (AE) which is iteratively trained on incoming data. The scoring function uses the reconstruction error and latent space information. A model of the identified subsequence is saved and used for recognition of repeating subsequences as well as fast offline clustering. For evaluation, we propose a new similarity measure based on the curvature for a more intuitive time-series subsequence clustering metric. A comparison with seven other state-of-the-art algorithms and eight datasets shows the capability and the increased performance of our algorithm to cluster MTSD online and offline in conjunction with mechatronic systems.Comment: 26 pages, 11 figures, for associated python code repositories see https://github.com/Jokonu/mt3scm and https://github.com/Jokonu/abimca; Minor spelling and grammar corrections, fixed wrong bibtex entry for SOStream, some improvements and corrections in formulas of section

Decentralized Vision-Based Byzantine Agent Detection in Multi-Robot Systems with IOTA Smart Contracts

Multiple opportunities lie at the intersection of multi-robot systems and distributed ledger technologies (DLTs). In this work, we investigate the potential of new DLT solutions such as IOTA, for detecting anomalies and byzantine agents in multi-robot systems in a decentralized manner. Traditional blockchain approaches are not applicable to real-world networked and decentralized robotic systems where connectivity conditions are not ideal. To address this, we leverage recent advances in partition-tolerant and byzantine-tolerant collaborative decision-making processes with IOTA smart contracts. We show how our work in vision-based anomaly and change detection can be applied to detecting byzantine agents within multiple robots operating in the same environment. We show that IOTA smart contracts add a low computational overhead while allowing to build trust within the multi-robot system. The proposed approach effectively enables byzantine robot detection based on the comparison of images submitted by the different robots and detection of anomalies and changes between them

Adversarial Data Augmentation for HMM-based Anomaly Detection

In this work, we concentrate on the detection of anomalous behaviors in systems operating in the physical world and for which it is usually not possible to have a complete set of all possible anomalies in advance. We present a data augmentation and retraining approach based on adversarial learning for improving anomaly detection. In particular, we first define a method for gener- ating adversarial examples for anomaly detectors based on Hidden Markov Models (HMMs). Then, we present a data augmentation and retraining technique that uses these adversarial examples to improve anomaly detection performance. Finally, we evaluate our adversarial data augmentation and retraining approach on four datasets showing that it achieves a statistically significant perfor- mance improvement and enhances the robustness to adversarial attacks. Key differences from the state-of-the-art on adversarial data augmentation are the focus on multivariate time series (as opposed to images), the context of one-class classification (in contrast to standard multi-class classification), and the use of HMMs (in contrast to neural networks)

Spatiotemporal anomaly detection: streaming architecture and algorithms

Includes bibliographical references.2020 Summer.Anomaly detection is the science of identifying one or more rare or unexplainable samples or events in a dataset or data stream. The field of anomaly detection has been extensively studied by mathematicians, statisticians, economists, engineers, and computer scientists. One open research question remains the design of distributed cloud-based architectures and algorithms that can accurately identify anomalies in previously unseen, unlabeled streaming, multivariate spatiotemporal data. With streaming data, time is of the essence, and insights are perishable. Real-world streaming spatiotemporal data originate from many sources, including mobile phones, supervisory control and data acquisition enabled (SCADA) devices, the internet-of-things (IoT), distributed sensor networks, and social media. Baseline experiments are performed on four (4) non-streaming, static anomaly detection multivariate datasets using unsupervised offline traditional machine learning (TML), and unsupervised neural network techniques. Multiple architectures, including autoencoders, generative adversarial networks, convolutional networks, and recurrent networks, are adapted for experimentation. Extensive experimentation demonstrates that neural networks produce superior detection accuracy over TML techniques. These same neural network architectures can be extended to process unlabeled spatiotemporal streaming using online learning. Space and time relationships are further exploited to provide additional insights and increased anomaly detection accuracy. A novel domain-independent architecture and set of algorithms called the Spatiotemporal Anomaly Detection Environment (STADE) is formulated. STADE is based on federated learning architecture. STADE streaming algorithms are based on a geographically unique, persistently executing neural networks using online stochastic gradient descent (SGD). STADE is designed to be pluggable, meaning that alternative algorithms may be substituted or combined to form an ensemble. STADE incorporates a Stream Anomaly Detector (SAD) and a Federated Anomaly Detector (FAD). The SAD executes at multiple locations on streaming data, while the FAD executes at a single server and identifies global patterns and relationships among the site anomalies. Each STADE site streams anomaly scores to the centralized FAD server for further spatiotemporal dependency analysis and logging. The FAD is based on recent advances in DNN-based federated learning. A STADE testbed is implemented to facilitate globally distributed experimentation using low-cost, commercial cloud infrastructure provided by Microsoft™. STADE testbed sites are situated in the cloud within each continent: Africa, Asia, Australia, Europe, North America, and South America. Communication occurs over the commercial internet. Three STADE case studies are investigated. The first case study processes commercial air traffic flows, the second case study processes global earthquake measurements, and the third case study processes social media (i.e., Twitter™) feeds. These case studies confirm that STADE is a viable architecture for the near real-time identification of anomalies in streaming data originating from (possibly) computationally disadvantaged, geographically dispersed sites. Moreover, the addition of the FAD provides enhanced anomaly detection capability. Since STADE is domain-independent, these findings can be easily extended to additional application domains and use cases

Deep Learning Techniques for Radar-Based Continuous Human Activity Recognition

Human capability to perform routine tasks declines with age and age-related problems. Remote human activity recognition (HAR) is beneficial for regular monitoring of the elderly population. This paper addresses the problem of the continuous detection of daily human activities using a mm-wave Doppler radar. In this study, two strategies have been employed: the first method uses un-equalized series of activities, whereas the second method utilizes a gradient-based strategy for equalization of the series of activities. The dynamic time warping (DTW) algorithm and Long Short-term Memory (LSTM) techniques have been implemented for the classification of un-equalized and equalized series of activities, respectively. The input for DTW was provided using three strategies. The first approach uses the pixel-level data of frames (UnSup-PLevel). In the other two strategies, a convolutional variational autoencoder (CVAE) is used to extract Un-Supervised Encoded features (UnSup-EnLevel) and Supervised Encoded features (Sup-EnLevel) from the series of Doppler frames. The second approach for equalized data series involves the application of four distinct feature extraction methods: i.e., convolutional neural networks (CNN), supervised and unsupervised CVAE, and principal component Analysis (PCA). The extracted features were considered as an input to the LSTM. This paper presents a comparative analysis of a novel supervised feature extraction pipeline, employing Sup-ENLevel-DTW and Sup-EnLevel-LSTM, against several state-of-the-art unsupervised methods, including UnSUp-EnLevel-DTW, UnSup-EnLevel-LSTM, CNN-LSTM, and PCA-LSTM. The results demonstrate the superiority of the Sup-EnLevel-LSTM strategy. However, the UnSup-PLevel strategy worked surprisingly well without using annotations and frame equalization