590 research outputs found
Anomaly detection in network traffic using dynamic graph mining with a sparse autoencoder.
Network based attacks on ecommerce websites can have serious economic consequences. Hence, anomaly detection in dynamic network traffic has become an increasingly important research topic in recent years. This paper proposes a novel dynamic Graph and sparse Autoencoder based Anomaly Detection algorithm named GAAD. In GAAD, the network traffic over contiguous time intervals is first modelled as a series of dynamic bipartite graph increments. One mode projection is performed on each bipartite graph increment and the adjacency matrix derived. Columns of the resultant adjacency matrix are then used to train a sparse autoencoder to reconstruct it. The sum of squared errors between the reconstructed approximation and original adjacency matrix is then calculated. An online learning algorithm is then used to estimate a Gaussian distribution that models the error distribution. Outlier error values are deemed to represent anomalous traffic flows corresponding to possible attacks. In the experiment, a network emulator was used to generate representative ecommerce traffic flows over a time period of 225 minutes with five attacks injected, including SYN scans, host emulation and DDoS attacks. ROC curves were generated to investigate the influence of the autoencoder hyper-parameters. It was found that increasing the number of hidden nodes and their activation level, and increasing sparseness resulted in improved performance. Analysis showed that the sparse autoencoder was unable to encode the highly structured adjacency matrix structures associated with attacks, hence they were detected as anomalies. In contrast, SVD and variants, such as the compact matrix decomposition, were found to accurately encode the attack matrices, hence they went undetected
Quick survey of graph-based fraud detection methods
In general, anomaly detection is the problem of distinguishing between normal
data samples with well defined patterns or signatures and those that do not
conform to the expected profiles. Financial transactions, customer reviews,
social media posts are all characterized by relational information. In these
networks, fraudulent behaviour may appear as a distinctive graph edge, such as
spam message, a node or a larger subgraph structure, such as when a group of
clients engage in money laundering schemes. Most commonly, these networks are
represented as attributed graphs, with numerical features complementing
relational information. We present a survey on anomaly detection techniques
used for fraud detection that exploit both the graph structure underlying the
data and the contextual information contained in the attributes
Unsupervised Intrusion Detection with Cross-Domain Artificial Intelligence Methods
Cybercrime is a major concern for corporations, business owners, governments and citizens, and it continues to grow in spite of increasing investments in security and fraud prevention. The main challenges in this research field are: being able to detect unknown attacks, and reducing the false positive ratio. The aim of this research work was to target both problems by leveraging four artificial intelligence techniques.
The first technique is a novel unsupervised learning method based on skip-gram modeling. It was designed, developed and tested against a public dataset with popular intrusion patterns. A high accuracy and a low false positive rate were achieved without prior knowledge of attack patterns.
The second technique is a novel unsupervised learning method based on topic modeling. It was applied to three related domains (network attacks, payments fraud, IoT malware traffic). A high accuracy was achieved in the three scenarios, even though the malicious activity significantly differs from one domain to the other.
The third technique is a novel unsupervised learning method based on deep autoencoders, with feature selection performed by a supervised method, random forest. Obtained results showed that this technique can outperform other similar techniques.
The fourth technique is based on an MLP neural network, and is applied to alert reduction in fraud prevention. This method automates manual reviews previously done by human experts, without significantly impacting accuracy
Graph learning for anomaly analytics : algorithms, applications, and challenges
Anomaly analytics is a popular and vital task in various research contexts that has been studied for several decades. At the same time, deep learning has shown its capacity in solving many graph-based tasks, like node classification, link prediction, and graph classification. Recently, many studies are extending graph learning models for solving anomaly analytics problems, resulting in beneficial advances in graph-based anomaly analytics techniques. In this survey, we provide a comprehensive overview of graph learning methods for anomaly analytics tasks. We classify them into four categories based on their model architectures, namely graph convolutional network, graph attention network, graph autoencoder, and other graph learning models. The differences between these methods are also compared in a systematic manner. Furthermore, we outline several graph-based anomaly analytics applications across various domains in the real world. Finally, we discuss five potential future research directions in this rapidly growing field. © 2023 Association for Computing Machinery
Graph Learning for Anomaly Analytics: Algorithms, Applications, and Challenges
Anomaly analytics is a popular and vital task in various research contexts,
which has been studied for several decades. At the same time, deep learning has
shown its capacity in solving many graph-based tasks like, node classification,
link prediction, and graph classification. Recently, many studies are extending
graph learning models for solving anomaly analytics problems, resulting in
beneficial advances in graph-based anomaly analytics techniques. In this
survey, we provide a comprehensive overview of graph learning methods for
anomaly analytics tasks. We classify them into four categories based on their
model architectures, namely graph convolutional network (GCN), graph attention
network (GAT), graph autoencoder (GAE), and other graph learning models. The
differences between these methods are also compared in a systematic manner.
Furthermore, we outline several graph-based anomaly analytics applications
across various domains in the real world. Finally, we discuss five potential
future research directions in this rapidly growing field
A Survey on Explainable Anomaly Detection
In the past two decades, most research on anomaly detection has focused on
improving the accuracy of the detection, while largely ignoring the
explainability of the corresponding methods and thus leaving the explanation of
outcomes to practitioners. As anomaly detection algorithms are increasingly
used in safety-critical domains, providing explanations for the high-stakes
decisions made in those domains has become an ethical and regulatory
requirement. Therefore, this work provides a comprehensive and structured
survey on state-of-the-art explainable anomaly detection techniques. We propose
a taxonomy based on the main aspects that characterize each explainable anomaly
detection technique, aiming to help practitioners and researchers find the
explainable anomaly detection method that best suits their needs.Comment: Paper accepted by the ACM Transactions on Knowledge Discovery from
Data (TKDD) for publication (preprint version
The New Abnormal: Network Anomalies in the AI Era
Anomaly detection aims at finding unexpected patterns in data. It has been used in several problems in computer networks, from the detection of port scans and DDoS attacks to the monitoring of time-series collected from Internet monitoring systems. Data-driven approaches and machine learning have seen widespread application on anomaly detection too, and this trend has been accelerated by the recent developments on Artificial Intelligence research. This chapter summarizes ongoing recent progresses on anomaly detection research. In particular, we evaluate how developments on AI algorithms bring new possibilities for anomaly detection. We cover new representation learning techniques such as Generative Artificial Networks and Autoencoders, as well as techniques that can be used to improve models learned with machine learning algorithms, such as reinforcement learning. We survey both research works and tools implementing AI algorithms for anomaly detection. We found that the novel algorithms, while successful in other fields, have hardly been applied to networking problems. We conclude the chapter with a case study that illustrates a possible research direction
Graph Anomaly Detection with Graph Neural Networks: Current Status and Challenges
Graphs are used widely to model complex systems, and detecting anomalies in a
graph is an important task in the analysis of complex systems. Graph anomalies
are patterns in a graph that do not conform to normal patterns expected of the
attributes and/or structures of the graph. In recent years, graph neural
networks (GNNs) have been studied extensively and have successfully performed
difficult machine learning tasks in node classification, link prediction, and
graph classification thanks to the highly expressive capability via message
passing in effectively learning graph representations. To solve the graph
anomaly detection problem, GNN-based methods leverage information about the
graph attributes (or features) and/or structures to learn to score anomalies
appropriately. In this survey, we review the recent advances made in detecting
graph anomalies using GNN models. Specifically, we summarize GNN-based methods
according to the graph type (i.e., static and dynamic), the anomaly type (i.e.,
node, edge, subgraph, and whole graph), and the network architecture (e.g.,
graph autoencoder, graph convolutional network). To the best of our knowledge,
this survey is the first comprehensive review of graph anomaly detection
methods based on GNNs.Comment: 9 pages, 2 figures, 1 tables; to appear in the IEEE Access (Please
cite our journal version.
- …