27,936 research outputs found
Deep Representation Learning for Social Network Analysis
Social network analysis is an important problem in data mining. A fundamental
step for analyzing social networks is to encode network data into
low-dimensional representations, i.e., network embeddings, so that the network
topology structure and other attribute information can be effectively
preserved. Network representation leaning facilitates further applications such
as classification, link prediction, anomaly detection and clustering. In
addition, techniques based on deep neural networks have attracted great
interests over the past a few years. In this survey, we conduct a comprehensive
review of current literature in network representation learning utilizing
neural network models. First, we introduce the basic models for learning node
representations in homogeneous networks. Meanwhile, we will also introduce some
extensions of the base models in tackling more complex scenarios, such as
analyzing attributed networks, heterogeneous networks and dynamic networks.
Then, we introduce the techniques for embedding subgraphs. After that, we
present the applications of network representation learning. At the end, we
discuss some promising research directions for future work
N-BaIoT: Network-based Detection of IoT Botnet Attacks Using Deep Autoencoders
The proliferation of IoT devices which can be more easily compromised than
desktop computers has led to an increase in the occurrence of IoT based botnet
attacks. In order to mitigate this new threat there is a need to develop new
methods for detecting attacks launched from compromised IoT devices and
differentiate between hour and millisecond long IoTbased attacks. In this paper
we propose and empirically evaluate a novel network based anomaly detection
method which extracts behavior snapshots of the network and uses deep
autoencoders to detect anomalous network traffic emanating from compromised IoT
devices. To evaluate our method, we infected nine commercial IoT devices in our
lab with two of the most widely known IoT based botnets, Mirai and BASHLITE.
Our evaluation results demonstrated our proposed method's ability to accurately
and instantly detect the attacks as they were being launched from the
compromised IoT devices which were part of a botnet.Comment: Accepted for publication in July September issue of IEEE Pervasive
Computin
Should I Raise The Red Flag? A comprehensive survey of anomaly scoring methods toward mitigating false alarms
Nowadays, advanced intrusion detection systems (IDSs) rely on a combination
of anomaly detection and signature-based methods. An IDS gathers observations,
analyzes behavioral patterns, and reports suspicious events for further
investigation. A notorious issue anomaly detection systems (ADSs) and IDSs face
is the possibility of high false alarms, which even state-of-the-art systems
have not overcome. This is especially a problem with large and complex systems.
The number of non-critical alarms can easily overwhelm administrators and
increase the likelihood of ignoring future alerts. Mitigation strategies thus
aim to avoid raising `too many' false alarms without missing potentially
dangerous situations. There are two major categories of false alarm-mitigation
strategies: (1) methods that are customized to enhance the quality of anomaly
scoring; (2) approaches acting as filtering methods in contexts that aim to
decrease false alarm rates. These methods have been widely utilized by many
scholars. Herein, we review and compare the existing techniques for false alarm
mitigation in ADSs. We also examine the use of promising techniques in
signature-based IDS and other relevant contexts, such as commercial security
information and event management tools, which are promising for ADSs. We
conclude by highlighting promising directions for future research.Comment: arXiv admin note: text overlap with arXiv:1802.04431,
arXiv:1503.01158 by other author
Finding Rats in Cats: Detecting Stealthy Attacks using Group Anomaly Detection
Advanced attack campaigns span across multiple stages and stay stealthy for
long time periods. There is a growing trend of attackers using off-the-shelf
tools and pre-installed system applications (such as \emph{powershell} and
\emph{wmic}) to evade the detection because the same tools are also used by
system administrators and security analysts for legitimate purposes for their
routine tasks. To start investigations, event logs can be collected from
operational systems; however, these logs are generic enough and it often
becomes impossible to attribute a potential attack to a specific attack group.
Recent approaches in the literature have used anomaly detection techniques,
which aim at distinguishing between malicious and normal behavior of computers
or network systems. Unfortunately, anomaly detection systems based on point
anomalies are too rigid in a sense that they could miss the malicious activity
and classify the attack, not an outlier. Therefore, there is a research
challenge to make better detection of malicious activities. To address this
challenge, in this paper, we leverage Group Anomaly Detection (GAD), which
detects anomalous collections of individual data points.
Our approach is to build a neural network model utilizing Adversarial
Autoencoder (AAE-) in order to detect the activity of an attacker who
leverages off-the-shelf tools and system applications. In addition, we also
build \textit{Behavior2Vec} and \textit{Command2Vec} sentence embedding deep
learning models specific for feature extraction tasks. We conduct extensive
experiments to evaluate our models on real-world datasets collected for a
period of two months. The empirical results demonstrate that our approach is
effective and robust in discovering targeted attacks, pen-tests, and attack
campaigns leveraging custom tools.Comment: Preprint: Modified, Extended Version will be presented at TrustCom
201
Anomaly Detection in Images
Visual defect assessment is a form of anomaly detection. This is very
relevant in finding faults such as cracks and markings in various surface
inspection tasks like pavement and automotive parts. The task involves
detection of deviation/divergence of anomalous samples from the normal ones.
Two of the major challenges in supervised anomaly detection are the lack of
labelled training data and the low availability of anomaly instances.
Semi-supervised methods which learn the underlying distribution of the normal
samples and then measure the deviation/divergence from the estimated model as
the anomaly score have limitations in their overall ability to detect
anomalies. This paper proposes the application of network-based deep transfer
learning using convolutional neural networks (CNNs) for the task of anomaly
detection. Single class SVMs have been used in the past with some success,
however we hypothesize that deeper networks for single class classification
should perform better. Results obtained on established anomaly detection
benchmarks as well as on a real-world dataset, show that the proposed method
clearly outperforms the existing state-of-the-art methods, by achieving a
staggering average area under the receiver operating characteristic curve value
of 0.99 for the tested data-sets which is an average improvement of 41% on the
CIFAR10, 20% on MNIST and 16% on Cement Crack data-sets
Anomaly Detection using Deep Learning based Image Completion
Automated surface inspection is an important task in many manufacturing
industries and often requires machine learning driven solutions. Supervised
approaches, however, can be challenging, since it is often difficult to obtain
large amounts of labeled training data. In this work, we instead perform
one-class unsupervised learning on fault-free samples by training a deep
convolutional neural network to complete images whose center regions are cut
out. Since the network is trained exclusively on fault-free data, it completes
the image patches with a fault-free version of the missing image region. The
pixel-wise reconstruction error within the cut out region is an anomaly image
which can be used for anomaly detection. Results on surface images of decorated
plastic parts demonstrate that this approach is suitable for detection of
visible anomalies and moreover surpasses all other tested methods.Comment: 6 pages, 5 figures, Accepted for publication by IEEE, 17th
International Conference on Machine Learning and Applications (ICMLA) 201
A Survey on Unknown Presentation Attack Detection for Fingerprint
Fingerprint recognition systems are widely deployed in various real-life
applications as they have achieved high accuracy. The widely used applications
include border control, automated teller machine (ATM), and attendance
monitoring systems. However, these critical systems are prone to spoofing
attacks (a.k.a presentation attacks (PA)). PA for fingerprint can be performed
by presenting gummy fingers made from different materials such as silicone,
gelatine, play-doh, ecoflex, 2D printed paper, 3D printed material, or latex.
Biometrics Researchers have developed Presentation Attack Detection (PAD)
methods as a countermeasure to PA. PAD is usually done by training a machine
learning classifier for known attacks for a given dataset, and they achieve
high accuracy in this task. However, generalizing to unknown attacks is an
essential problem from applicability to real-world systems, mainly because
attacks cannot be exhaustively listed in advance. In this survey paper, we
present a comprehensive survey on existing PAD algorithms for fingerprint
recognition systems, specifically from the standpoint of detecting unknown PAD.
We categorize PAD algorithms, point out their advantages/disadvantages, and
future directions for this area.Comment: Submitted to 3rd International Conference on Intelligent Technologies
and Applications INTAP 202
Detection of Anomalies in Large Scale Accounting Data using Deep Autoencoder Networks
Learning to detect fraud in large-scale accounting data is one of the
long-standing challenges in financial statement audits or fraud investigations.
Nowadays, the majority of applied techniques refer to handcrafted rules derived
from known fraud scenarios. While fairly successful, these rules exhibit the
drawback that they often fail to generalize beyond known fraud scenarios and
fraudsters gradually find ways to circumvent them. To overcome this
disadvantage and inspired by the recent success of deep learning we propose the
application of deep autoencoder neural networks to detect anomalous journal
entries. We demonstrate that the trained network's reconstruction error
obtainable for a journal entry and regularized by the entry's individual
attribute probabilities can be interpreted as a highly adaptive anomaly
assessment. Experiments on two real-world datasets of journal entries, show the
effectiveness of the approach resulting in high f1-scores of 32.93 (dataset A)
and 16.95 (dataset B) and less false positive alerts compared to state of the
art baseline methods. Initial feedback received by chartered accountants and
fraud examiners underpinned the quality of the approach in capturing highly
relevant accounting anomalies.Comment: 19 pages, 6 figures, 3 table
Transformation Based Deep Anomaly Detection in Astronomical Images
In this work, we propose several enhancements to a geometric transformation
based model for anomaly detection in images (GeoTranform). The model assumes
that the anomaly class is unknown and that only inlier samples are available
for training. We introduce new filter based transformations useful for
detecting anomalies in astronomical images, that highlight artifact properties
to make them more easily distinguishable from real objects. In addition, we
propose a transformation selection strategy that allows us to find
indistinguishable pairs of transformations. This results in an improvement of
the area under the Receiver Operating Characteristic curve (AUROC) and accuracy
performance, as well as in a dimensionality reduction. The models were tested
on astronomical images from the High Cadence Transient Survey (HiTS) and Zwicky
Transient Facility (ZTF) datasets. The best models obtained an average AUROC of
99.20% for HiTS and 91.39% for ZTF. The improvement over the original
GeoTransform algorithm and baseline methods such as One-Class Support Vector
Machine, and deep learning based methods is significant both statistically and
in practice.Comment: 8 pages, 6 figures, 4 tables. Accepted for publication in proceedings
of the IEEE World Congress on Computational Intelligence (IEEE WCCI),
Glasgow, UK, 19-24 July, 202
Fixing Bias in Reconstruction-based Anomaly Detection with Lipschitz Discriminators
Anomaly detection is of great interest in fields where abnormalities need to
be identified and corrected (e.g., medicine and finance). Deep learning methods
for this task often rely on autoencoder reconstruction error, sometimes in
conjunction with other errors. We show that this approach exhibits intrinsic
biases that lead to undesirable results. Reconstruction-based methods are
sensitive to training-data outliers and simple-to-reconstruct points. Instead,
we introduce a new unsupervised Lipschitz anomaly discriminator that does not
suffer from these biases. Our anomaly discriminator is trained, similar to the
ones used in GANs, to detect the difference between the training data and
corruptions of the training data. We show that this procedure successfully
detects unseen anomalies with guarantees on those that have a certain
Wasserstein distance from the data or corrupted training set. These additions
allow us to show improved performance on MNIST, CIFAR10, and health record
data.Comment: 6 pages, 4 figures, 2 tables, presented at IEEE MLS
- …