15,239 research outputs found
Anomaly Detection and Localization in NFV Systems: an Unsupervised Learning Approach
Due to the scarcity of labeled faulty data, Unsupervised Learning (UL) methods have gained great traction for anomaly detection and localization in Network Functions Virtualization (NFV) systems. In a UL approach, training is performed on only normal data for learning normal data patterns, and deviation from the norm is considered as an anomaly. However, it has been shown that even small percentages of anomalous samples in the training data (referred to as contamination) can significantly degrade the performance of UL methods. To address this issue, we propose an anomaly-detection approach based on the Noisy-Student technique, which was originally introduced for leveraging unlabeled datasets in computer-vision classification problems. Our approach not only provides robustness against training-data contamination, but also can leverage this contamination to improve anomaly-detection accuracy. Moreover, after an anomaly is detected, localization of the anomalous virtualized network functions in an unsupervised manner is a challenging task in the absence of labeled data. For anomaly localization in NFV systems, we propose to exploit existing local AI-explainability methods to achieve a high localization performance and propose our own novel AI-explainability method, specifically designed for the anomaly-localization problem in NFV, to improve the performance further. We perform a comprehensive experimental analysis on two datasets collected on different NFV testbeds and show that our proposed solutions outperform the existing methods by up to 22% in anomaly detection and up to 19% in anomaly localization in terms of F1-score
Network-Wide Traffic Anomaly Detection and Localization Based on Robust Multivariate Probabilistic Calibration Model
Network anomaly detection and localization are of great significance to network security. Compared with the traditional methods of host computer, single link and single path, the network-wide anomaly detection approaches have distinctive advantages with respect to detection precision and range. However, when facing the actual problems of noise interference or data loss, the network-wide anomaly detection approaches also suffer significant performance reduction or may even become unavailable. Besides, researches on anomaly localization are rare. In order to solve the mentioned problems, this paper presents a robust multivariate probabilistic calibration model for network-wide anomaly detection and localization. It applies the latent variable probability theory with multivariate t-distribution to establish the normal traffic model. Not only does the algorithm implement network anomaly detection by judging whether the sample’s Mahalanobis distance exceeds the threshold, but also it locates anomalies by contribution analysis. Both theoretical analysis and experimental results demonstrate its robustness and wider use. The algorithm is applicable when dealing with both data integrity and loss. It also has a stronger resistance over noise interference and lower sensitivity to the change of parameters, all of which indicate its performance stability
Multiresolution Feature Guidance Based Transformer for Anomaly Detection
Anomaly detection is represented as an unsupervised learning to identify
deviated images from normal images. In general, there are two main challenges
of anomaly detection tasks, i.e., the class imbalance and the unexpectedness of
anomalies. In this paper, we propose a multiresolution feature guidance method
based on Transformer named GTrans for unsupervised anomaly detection and
localization. In GTrans, an Anomaly Guided Network (AGN) pre-trained on
ImageNet is developed to provide surrogate labels for features and tokens.
Under the tacit knowledge guidance of the AGN, the anomaly detection network
named Trans utilizes Transformer to effectively establish a relationship
between features with multiresolution, enhancing the ability of the Trans in
fitting the normal data manifold. Due to the strong generalization ability of
AGN, GTrans locates anomalies by comparing the differences in spatial distance
and direction of multi-scale features extracted from the AGN and the Trans. Our
experiments demonstrate that the proposed GTrans achieves state-of-the-art
performance in both detection and localization on the MVTec AD dataset. GTrans
achieves image-level and pixel-level anomaly detection AUROC scores of 99.0%
and 97.9% on the MVTec AD dataset, respectively
A Family of Joint Sparse PCA Algorithms for Anomaly Localization in Network Data Streams
Determining anomalies in data streams that are collected and transformed from various types of networks has recently attracted significant research interest. Principal Component Analysis (PCA) is arguably the most widely applied unsupervised anomaly detection technique for networked data streams due to its simplicity and efficiency. However, none of existing PCA based approaches addresses the problem of identifying the sources that contribute most to the observed anomaly, or anomaly localization. In this paper, we first proposed a novel joint sparse PCA method to perform anomaly detection and localization for network data streams. Our key observation is that we can detect anomalies and localize anomalous sources by identifying a low dimensional abnormal subspace that captures the abnormal behavior of data. To better capture the sources of anomalies, we incorporated the structure of the network stream data in our anomaly localization framework. Also, an extended version of PCA, multidimensional KLE, was introduced to stabilize the localization performance. We performed comprehensive experimental studies on four real-world data sets from different application domains and compared our proposed techniques with several state-of-the-arts. Our experimental studies demonstrate the utility of the proposed methods
Image Anomaly Detection and Localization with Position and Neighborhood Information
Anomaly detection and localization are essential in many areas, where
collecting enough anomalous samples for training is almost impossible. To
overcome this difficulty, many existing methods use a pre-trained network to
encode input images and non-parametric modeling to estimate the encoded feature
distribution. In the modeling process, however, they overlook that position and
neighborhood information affect the distribution of normal features. To use the
information, in this paper, the normal distribution is estimated with
conditional probability given neighborhood features, which is modeled with a
multi-layer perceptron network. At the same time, positional information can be
used by building a histogram of representative features at each position. While
existing methods simply resize the anomaly map into the resolution of an input
image, the proposed method uses an additional refine network that is trained
from synthetic anomaly images to perform better interpolation considering the
shape and edge of the input image. For the popular industrial dataset, MVTec AD
benchmark, the experimental results show \textbf{99.52\%} and \textbf{98.91\%}
AUROC scores in anomaly detection and localization, which is state-of-the-art
performance
Structural Teacher-Student Normality Learning for Multi-Class Anomaly Detection and Localization
Visual anomaly detection is a challenging open-set task aimed at identifying
unknown anomalous patterns while modeling normal data. The knowledge
distillation paradigm has shown remarkable performance in one-class anomaly
detection by leveraging teacher-student network feature comparisons. However,
extending this paradigm to multi-class anomaly detection introduces novel
scalability challenges. In this study, we address the significant performance
degradation observed in previous teacher-student models when applied to
multi-class anomaly detection, which we identify as resulting from cross-class
interference. To tackle this issue, we introduce a novel approach known as
Structural Teacher-Student Normality Learning (SNL): (1) We propose
spatial-channel distillation and intra-&inter-affinity distillation techniques
to measure structural distance between the teacher and student networks. (2) We
introduce a central residual aggregation module (CRAM) to encapsulate the
normal representation space of the student network. We evaluate our proposed
approach on two anomaly detection datasets, MVTecAD and VisA. Our method
surpasses the state-of-the-art distillation-based algorithms by a significant
margin of 3.9% and 1.5% on MVTecAD and 1.2% and 2.5% on VisA in the multi-class
anomaly detection and localization tasks, respectively. Furthermore, our
algorithm outperforms the current state-of-the-art unified models on both
MVTecAD and VisA
Anomaly Detection in UASN Localization Based on Time Series Analysis and Fuzzy Logic
[EN] Underwater acoustic sensor network (UASN) offers a promising solution for exploring underwater resources remotely. For getting a better understanding of sensed data, accurate localization is essential. As the UASN acoustic channel is open and the environment is hostile, the risk of malicious activities is very high, particularly in time-critical military applications. Since the location estimation with false data ends up in wrong positioning, it is necessary to identify and ignore such data to ensure data integrity. Therefore, in this paper, we propose a novel anomaly detection system for UASN localization. To minimize computational power and storage, we designed separate anomaly detection schemes for sensor nodes and anchor nodes. We propose an auto-regressive prediction-based scheme for detecting anomalies at sensor nodes. For anchor nodes, a fuzzy inference system is designed to identify the presence of anomalous behavior. The detection schemes are implemented at every node for enabling identification of multiple and duplicate anomalies at its origin. We simulated the network, modeled anomalies and analyzed the performance of detection schemes at anchor nodes and sensor nodes. The results indicate that anomaly detection systems offer an acceptable accuracy with high true positive rate and F-Score.Das, AP.; Thampi, SM.; Lloret, J. (2020). Anomaly Detection in UASN Localization Based on Time Series Analysis and Fuzzy Logic. Mobile Networks and Applications (Online). 25(1):55-67. https://doi.org/10.1007/s11036-018-1192-y556725
An initial approach to distributed adaptive fault-handling in networked systems
We present a distributed adaptive fault-handling algorithm applied in networked systems. The probabilistic approach that we use makes the proposed method capable of adaptively detect and localize network faults by the use of simple end-to-end test transactions. Our method operates in a fully distributed manner, such that each network element detects faults using locally extracted information as input. This allows for a fast autonomous adaption to local network conditions in real-time, with significantly reduced need for manual configuration of algorithm parameters. Initial results from a small synthetically generated network indicate that satisfactory algorithm performance can be achieved, with respect to the number of detected and localized faults, detection time and false alarm rate
Detection and localization of change-points in high-dimensional network traffic data
We propose a novel and efficient method, that we shall call TopRank in the
following paper, for detecting change-points in high-dimensional data. This
issue is of growing concern to the network security community since network
anomalies such as Denial of Service (DoS) attacks lead to changes in Internet
traffic. Our method consists of a data reduction stage based on record
filtering, followed by a nonparametric change-point detection test based on
-statistics. Using this approach, we can address massive data streams and
perform anomaly detection and localization on the fly. We show how it applies
to some real Internet traffic provided by France-T\'el\'ecom (a French Internet
service provider) in the framework of the ANR-RNRT OSCAR project. This approach
is very attractive since it benefits from a low computational load and is able
to detect and localize several types of network anomalies. We also assess the
performance of the TopRank algorithm using synthetic data and compare it with
alternative approaches based on random aggregation.Comment: Published in at http://dx.doi.org/10.1214/08-AOAS232 the Annals of
Applied Statistics (http://www.imstat.org/aoas/) by the Institute of
Mathematical Statistics (http://www.imstat.org
- …