2,005 research outputs found
Deep Structured Energy Based Models for Anomaly Detection
In this paper, we attack the anomaly detection problem by directly modeling
the data distribution with deep architectures. We propose deep structured
energy based models (DSEBMs), where the energy function is the output of a
deterministic deep neural network with structure. We develop novel model
architectures to integrate EBMs with different types of data such as static
data, sequential data, and spatial data, and apply appropriate model
architectures to adapt to the data structure. Our training algorithm is built
upon the recent development of score matching \cite{sm}, which connects an EBM
with a regularized autoencoder, eliminating the need for complicated sampling
method. Statistically sound decision criterion can be derived for anomaly
detection purpose from the perspective of the energy landscape of the data
distribution. We investigate two decision criteria for performing anomaly
detection: the energy score and the reconstruction error. Extensive empirical
studies on benchmark tasks demonstrate that our proposed model consistently
matches or outperforms all the competing methods.Comment: To appear in ICML 201
Robust, Deep and Inductive Anomaly Detection
PCA is a classical statistical technique whose simplicity and maturity has
seen it find widespread use as an anomaly detection technique. However, it is
limited in this regard by being sensitive to gross perturbations of the input,
and by seeking a linear subspace that captures normal behaviour. The first
issue has been dealt with by robust PCA, a variant of PCA that explicitly
allows for some data points to be arbitrarily corrupted, however, this does not
resolve the second issue, and indeed introduces the new issue that one can no
longer inductively find anomalies on a test set. This paper addresses both
issues in a single model, the robust autoencoder. This method learns a
nonlinear subspace that captures the majority of data points, while allowing
for some data to have arbitrary corruption. The model is simple to train and
leverages recent advances in the optimisation of deep neural networks.
Experiments on a range of real-world datasets highlight the model's
effectiveness.Comment: Accepted ECML PKDD 2017 Skopje, Macedonia 18-22 September the
European Conference On Machine Learning & Principles and Practice of
Knowledge Discover
Catching Anomalous Distributed Photovoltaics: An Edge-based Multi-modal Anomaly Detection
A significant challenge in energy system cyber security is the current
inability to detect cyber-physical attacks targeting and originating from
distributed grid-edge devices such as photovoltaics (PV) panels, smart flexible
loads, and electric vehicles. We address this concern by designing and
developing a distributed, multi-modal anomaly detection approach that can sense
the health of the device and the electric power grid from the edge. This is
realized by exploiting unsupervised machine learning algorithms on multiple
sources of time-series data, fusing these multiple local observations and
flagging anomalies when a deviation from the normal behavior is observed.
We particularly focus on the cyber-physical threats to the distributed PVs
that has the potential to cause local disturbances or grid instabilities by
creating supply-demand mismatch, reverse power flow conditions etc. We use an
open source power system simulation tool called GridLAB-D, loaded with real
smart home and solar datasets to simulate the smart grid scenarios and to
illustrate the impact of PV attacks on the power system. Various attacks
targeting PV panels that create voltage fluctuations, reverse power flow etc
were designed and performed. We observe that while individual unsupervised
learning algorithms such as OCSVMs, Corrupt RF and PCA surpasses in identifying
particular attack type, PCA with Convex Hull outperforms all algorithms in
identifying all designed attacks with a true positive rate of 83.64% and an
accuracy of 95.78%. Our key insight is that due to the heterogeneous nature of
the distribution grid and the uncertainty in the type of the attack being
launched, relying on single mode of information for defense can lead to
increased false alarms and missed detection rates as one can design attacks to
hide within those uncertainties and remain stealthy
Adversarially Learned One-Class Classifier for Novelty Detection
Novelty detection is the process of identifying the observation(s) that
differ in some respect from the training observations (the target class). In
reality, the novelty class is often absent during training, poorly sampled or
not well defined. Therefore, one-class classifiers can efficiently model such
problems. However, due to the unavailability of data from the novelty class,
training an end-to-end deep network is a cumbersome task. In this paper,
inspired by the success of generative adversarial networks for training deep
models in unsupervised and semi-supervised settings, we propose an end-to-end
architecture for one-class classification. Our architecture is composed of two
deep networks, each of which trained by competing with each other while
collaborating to understand the underlying concept in the target class, and
then classify the testing samples. One network works as the novelty detector,
while the other supports it by enhancing the inlier samples and distorting the
outliers. The intuition is that the separability of the enhanced inliers and
distorted outliers is much better than deciding on the original samples. The
proposed framework applies to different related applications of anomaly and
outlier detection in images and videos. The results on MNIST and Caltech-256
image datasets, along with the challenging UCSD Ped2 dataset for video anomaly
detection illustrate that our proposed method learns the target class
effectively and is superior to the baseline and state-of-the-art methods.Comment: CVPR 2018 Pape
A Unifying Review of Deep and Shallow Anomaly Detection
Deep learning approaches to anomaly detection have recently improved the
state of the art in detection performance on complex datasets such as large
collections of images or text. These results have sparked a renewed interest in
the anomaly detection problem and led to the introduction of a great variety of
new methods. With the emergence of numerous such methods, including approaches
based on generative models, one-class classification, and reconstruction, there
is a growing need to bring methods of this field into a systematic and unified
perspective. In this review we aim to identify the common underlying principles
as well as the assumptions that are often made implicitly by various methods.
In particular, we draw connections between classic 'shallow' and novel deep
approaches and show how this relation might cross-fertilize or extend both
directions. We further provide an empirical assessment of major existing
methods that is enriched by the use of recent explainability techniques, and
present specific worked-through examples together with practical advice.
Finally, we outline critical open challenges and identify specific paths for
future research in anomaly detection.Comment: 40 pages; accepted for publication in the Proceedings of the IEEE
Estimation of Dimensions Contributing to Detected Anomalies with Variational Autoencoders
Anomaly detection using dimensionality reduction has been an essential
technique for monitoring multidimensional data. Although deep learning-based
methods have been well studied for their remarkable detection performance,
their interpretability is still a problem. In this paper, we propose a novel
algorithm for estimating the dimensions contributing to the detected anomalies
by using variational autoencoders (VAEs). Our algorithm is based on an
approximative probabilistic model that considers the existence of anomalies in
the data, and by maximizing the log-likelihood, we estimate which dimensions
contribute to determining data as an anomaly. The experiments results with
benchmark datasets show that our algorithm extracts the contributing dimensions
more accurately than baseline methods
Energy-based Models for Video Anomaly Detection
Automated detection of abnormalities in data has been studied in research
area in recent years because of its diverse applications in practice including
video surveillance, industrial damage detection and network intrusion
detection. However, building an effective anomaly detection system is a
non-trivial task since it requires to tackle challenging issues of the shortage
of annotated data, inability of defining anomaly objects explicitly and the
expensive cost of feature engineering procedure. Unlike existing appoaches
which only partially solve these problems, we develop a unique framework to
cope the problems above simultaneously. Instead of hanlding with ambiguous
definition of anomaly objects, we propose to work with regular patterns whose
unlabeled data is abundant and usually easy to collect in practice. This allows
our system to be trained completely in an unsupervised procedure and liberate
us from the need for costly data annotation. By learning generative model that
capture the normality distribution in data, we can isolate abnormal data points
that result in low normality scores (high abnormality scores). Moreover, by
leverage on the power of generative networks, i.e. energy-based models, we are
also able to learn the feature representation automatically rather than
replying on hand-crafted features that have been dominating anomaly detection
research over many decades. We demonstrate our proposal on the specific
application of video anomaly detection and the experimental results indicate
that our method performs better than baselines and are comparable with
state-of-the-art methods in many benchmark video anomaly detection datasets
Detection of Unknown Anomalies in Streaming Videos with Generative Energy-based Boltzmann Models
Abnormal event detection is one of the important objectives in research and
practical applications of video surveillance. However, there are still three
challenging problems for most anomaly detection systems in practical setting:
limited labeled data, ambiguous definition of "abnormal" and expensive feature
engineering steps. This paper introduces a unified detection framework to
handle these challenges using energy-based models, which are powerful tools for
unsupervised representation learning. Our proposed models are firstly trained
on unlabeled raw pixels of image frames from an input video rather than
hand-crafted visual features; and then identify the locations of abnormal
objects based on the errors between the input video and its reconstruction
produced by the models. To handle video stream, we develop an online version of
our framework, wherein the model parameters are updated incrementally with the
image frames arriving on the fly. Our experiments show that our detectors,
using Restricted Boltzmann Machines (RBMs) and Deep Boltzmann Machines (DBMs)
as core modules, achieve superior anomaly detection performance to unsupervised
baselines and obtain accuracy comparable with the state-of-the-art approaches
when evaluating at the pixel-level. More importantly, we discover that our
system trained with DBMs is able to simultaneously perform scene clustering and
scene reconstruction. This capacity not only distinguishes our method from
other existing detectors but also offers a unique tool to investigate and
understand how the model works.Comment: This manuscript is under consideration at Pattern Recognition Letter
Classification-Based Anomaly Detection for General Data
Anomaly detection, finding patterns that substantially deviate from those
seen previously, is one of the fundamental problems of artificial intelligence.
Recently, classification-based methods were shown to achieve superior results
on this task. In this work, we present a unifying view and propose an open-set
method, GOAD, to relax current generalization assumptions. Furthermore, we
extend the applicability of transformation-based methods to non-image data
using random affine transformations. Our method is shown to obtain
state-of-the-art accuracy and is applicable to broad data types. The strong
performance of our method is extensively validated on multiple datasets from
different domains.Comment: ICLR'2
One-Class Classification: A Survey
One-Class Classification (OCC) is a special case of multi-class
classification, where data observed during training is from a single positive
class. The goal of OCC is to learn a representation and/or a classifier that
enables recognition of positively labeled queries during inference. This topic
has received considerable amount of interest in the computer vision, machine
learning and biometrics communities in recent years. In this article, we
provide a survey of classical statistical and recent deep learning-based OCC
methods for visual recognition. We discuss the merits and drawbacks of existing
OCC approaches and identify promising avenues for research in this field. In
addition, we present a discussion of commonly used datasets and evaluation
metrics for OCC
- …