Scalable and Interpretable One-class SVMs with Deep Learning and Random Fourier features

One-class support vector machine (OC-SVM) for a long time has been one of the most effective anomaly detection methods and extensively adopted in both research as well as industrial applications. The biggest issue for OC-SVM is yet the capability to operate with large and high-dimensional datasets due to optimization complexity. Those problems might be mitigated via dimensionality reduction techniques such as manifold learning or autoencoder. However, previous work often treats representation learning and anomaly prediction separately. In this paper, we propose autoencoder based one-class support vector machine (AE-1SVM) that brings OC-SVM, with the aid of random Fourier features to approximate the radial basis kernel, into deep learning context by combining it with a representation learning architecture and jointly exploit stochastic gradient descent to obtain end-to-end training. Interestingly, this also opens up the possible use of gradient-based attribution methods to explain the decision making for anomaly detection, which has ever been challenging as a result of the implicit mappings between the input space and the kernel space. To the best of our knowledge, this is the first work to study the interpretability of deep learning in anomaly detection. We evaluate our method on a wide range of unsupervised anomaly detection tasks in which our end-to-end training architecture achieves a performance significantly better than the previous work using separate training.Comment: Accepted at European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD) 201

Multi-level damage detection using a combination of deep neural networks

In recent years, bridge damage identification using a convolutional neural network (CNN) has become a hot research topic and received much attention in the field of civil engineering. Although CNN is capable of categorizing damaged and undamaged states from the measured data, the level of accuracy for damage diagnosis is still insufficient due to the tendency of CNN to ignore the temporal dependency between data points. To address this problem, this paper introduces a novel hybrid damage detection method based on the combination of CNN and Long Short-Term Memory (LSTM) to classify and quantify different levels of damage in the bridge structure. In this method, the CNN model will be used to extract the spatial damage features, which will be combined with the temporal features obtained from Long Short-Term Memory (LSTM) model to create the enhanced damage features. The combination successfully strengthened the damage detection capability of the neural network. Moreover, deep learning is also improved in this paper to process the acceleration-time data, which has a different amplitude at short intervals and the same amplitude at long intervals. The empirical result on the Vang bridge shows that our hybrid CNN-LSTM can detect structural damage with a high level of accuracy