39,106 research outputs found
AIDA : Analytic isolation and distance-based anomaly detection algorithm
Many unsupervised anomaly detection algorithms rely on the concept of nearest neighbours to compute the anomaly scores. Such algorithms are popular because there are no assumptions about the data, making them a robust choice for unstructured datasets. However, the number (k) of nearest neighbours, which critically affects the model performance, cannot be tuned in an unsupervised setting. Hence, we propose the new and parameter-free Analytic Isolation and Distance-based Anomaly (AIDA) detection algorithm, that combines the metrics of distance with isolation. Based on AIDA, we also introduce the Tempered Isolation-based eXplanation (TIX) algorithm, which identifies the most relevant features characterizing an outlier, even in large multi-dimensional datasets, improving the overall explainability of the detection mechanism. Both AIDA and TIX are thoroughly tested and compared with state-of-the-art alternatives, proving to be useful additions to the existing set of tools in anomaly detection
AIDA : analytic isolation and distance-based anomaly detection algorithm
Many unsupervised anomaly detection algorithms rely on the concept of nearest neighbours to compute the anomaly scores. Such algorithms are popular because there are no assumptions about the data, making them a robust choice for unstructured datasets. However, the number (k) of nearest neighbours, which critically affects the model performance, cannot be tuned in an unsupervised setting. Hence, we propose the new and parameter-free Analytic Isolation and Distance-based Anomaly (AIDA) detection algorithm, that combines the metrics of distance with isolation. Based on AIDA, we also introduce the Tempered Isolation-based eXplanation (TIX) algorithm, which identifies the most relevant features characterizing an outlier, even in large multi-dimensional datasets, improving the overall explainability of the detection mechanism. Both AIDA and TIX are thoroughly tested and compared with state-of-the-art alternatives, proving to be useful additions to the existing set of tools in anomaly detection
OptIForest: Optimal Isolation Forest for Anomaly Detection
Anomaly detection plays an increasingly important role in various fields for
critical tasks such as intrusion detection in cybersecurity, financial risk
detection, and human health monitoring. A variety of anomaly detection methods
have been proposed, and a category based on the isolation forest mechanism
stands out due to its simplicity, effectiveness, and efficiency, e.g., iForest
is often employed as a state-of-the-art detector for real deployment. While the
majority of isolation forests use the binary structure, a framework LSHiForest
has demonstrated that the multi-fork isolation tree structure can lead to
better detection performance. However, there is no theoretical work answering
the fundamentally and practically important question on the optimal tree
structure for an isolation forest with respect to the branching factor. In this
paper, we establish a theory on isolation efficiency to answer the question and
determine the optimal branching factor for an isolation tree. Based on the
theoretical underpinning, we design a practical optimal isolation forest
OptIForest incorporating clustering based learning to hash which enables more
information to be learned from data for better isolation quality. The rationale
of our approach relies on a better bias-variance trade-off achieved by bias
reduction in OptIForest. Extensive experiments on a series of benchmarking
datasets for comparative and ablation studies demonstrate that our approach can
efficiently and robustly achieve better detection performance in general than
the state-of-the-arts including the deep learning based methods.Comment: This paper has been accepted by International Joint Conference on
Artificial Intelligence (IJCAI-23
Advanced Ground Systems Maintenance Enterprise Architecture Project
The project implements an architecture for delivery of integrated health management capabilities for the 21st Century launch complex. Capabilities include anomaly detection, fault isolation, prognostics and physics-based diagnostics
Advanced Ground Systems Maintenance Enterprise Architecture Project
The project implements an architecture for delivery of integrated health management capabilities for the 21st Century launch complex. The delivered capabilities include anomaly detection, fault isolation, prognostics and physics based diagnostics
- …