609 research outputs found
A Survey of Imbalanced Learning on Graphs: Problems, Techniques, and Future Directions
Graphs represent interconnected structures prevalent in a myriad of
real-world scenarios. Effective graph analytics, such as graph learning
methods, enables users to gain profound insights from graph data, underpinning
various tasks including node classification and link prediction. However, these
methods often suffer from data imbalance, a common issue in graph data where
certain segments possess abundant data while others are scarce, thereby leading
to biased learning outcomes. This necessitates the emerging field of imbalanced
learning on graphs, which aims to correct these data distribution skews for
more accurate and representative learning outcomes. In this survey, we embark
on a comprehensive review of the literature on imbalanced learning on graphs.
We begin by providing a definitive understanding of the concept and related
terminologies, establishing a strong foundational understanding for readers.
Following this, we propose two comprehensive taxonomies: (1) the problem
taxonomy, which describes the forms of imbalance we consider, the associated
tasks, and potential solutions; (2) the technique taxonomy, which details key
strategies for addressing these imbalances, and aids readers in their method
selection process. Finally, we suggest prospective future directions for both
problems and techniques within the sphere of imbalanced learning on graphs,
fostering further innovation in this critical area.Comment: The collection of awesome literature on imbalanced learning on
graphs: https://github.com/Xtra-Computing/Awesome-Literature-ILoG
Learning from Very Few Samples: A Survey
Few sample learning (FSL) is significant and challenging in the field of
machine learning. The capability of learning and generalizing from very few
samples successfully is a noticeable demarcation separating artificial
intelligence and human intelligence since humans can readily establish their
cognition to novelty from just a single or a handful of examples whereas
machine learning algorithms typically entail hundreds or thousands of
supervised samples to guarantee generalization ability. Despite the long
history dated back to the early 2000s and the widespread attention in recent
years with booming deep learning technologies, little surveys or reviews for
FSL are available until now. In this context, we extensively review 300+ papers
of FSL spanning from the 2000s to 2019 and provide a timely and comprehensive
survey for FSL. In this survey, we review the evolution history as well as the
current progress on FSL, categorize FSL approaches into the generative model
based and discriminative model based kinds in principle, and emphasize
particularly on the meta learning based FSL approaches. We also summarize
several recently emerging extensional topics of FSL and review the latest
advances on these topics. Furthermore, we highlight the important FSL
applications covering many research hotspots in computer vision, natural
language processing, audio and speech, reinforcement learning and robotic, data
analysis, etc. Finally, we conclude the survey with a discussion on promising
trends in the hope of providing guidance and insights to follow-up researches.Comment: 30 page
Self-Supervised Predictive Convolutional Attentive Block for Anomaly Detection
Anomaly detection is commonly pursued as a one-class classification problem,
where models can only learn from normal training samples, while being evaluated
on both normal and abnormal test samples. Among the successful approaches for
anomaly detection, a distinguished category of methods relies on predicting
masked information (e.g. patches, future frames, etc.) and leveraging the
reconstruction error with respect to the masked information as an abnormality
score. Different from related methods, we propose to integrate the
reconstruction-based functionality into a novel self-supervised predictive
architectural building block. The proposed self-supervised block is generic and
can easily be incorporated into various state-of-the-art anomaly detection
methods. Our block starts with a convolutional layer with dilated filters,
where the center area of the receptive field is masked. The resulting
activation maps are passed through a channel attention module. Our block is
equipped with a loss that minimizes the reconstruction error with respect to
the masked area in the receptive field. We demonstrate the generality of our
block by integrating it into several state-of-the-art frameworks for anomaly
detection on image and video, providing empirical evidence that shows
considerable performance improvements on MVTec AD, Avenue, and ShanghaiTech. We
release our code as open source at https://github.com/ristea/sspcab.Comment: Accepted at CVPR 2022. Paper + supplementary (14 pages, 9 figures
Unsupervised post-tuning of deep neural networks
International audienceWe propose in this work a new unsupervised training procedure that is most effective when it is applied after supervised training and fine-tuning of deep neural network classifiers. While standard regularization techniques combat overfitting by means that are unrelated to the target classification loss, such as by minimizing the L2 norm or by adding noise either in the data, model or process, the proposed unsupervised training loss reduces overfitting by optimizing the true classifier risk. The proposed approach is evaluated on several tasks of increasing difficulty and varying conditions: unsupervised training, posttuning and anomaly detection. It is also tested both on simple neural networks, such as small multi-layer perceptron, and complex Natural Language Processing models, e.g., pretrained BERT embeddings. Experimental results confirm the theory and show that the proposed approach gives the best results in posttuning conditions, i.e., when applied after supervised training and fine-tuning
PULL: Reactive Log Anomaly Detection Based On Iterative PU Learning
Due to the complexity of modern IT services, failures can be manifold, occur at any stage, and are hard to detect. For this reason, anomaly detection applied to monitoring data such as logs allows gaining relevant insights to improve IT services steadily and eradicate failures. However, existing anomaly detection methods that provide high accuracy often rely on labeled training data, which are time-consuming to obtain in practice. Therefore, we propose PULL, an iterative log analysis method for reactive anomaly detection based on estimated failure time windows provided by monitoring systems instead of labeled data. Our attention-based model uses a novel objective function for weak supervision deep learning that accounts for imbalanced data and applies an iterative learning strategy for positive and unknown samples (PU learning) to identify anomalous logs. Our evaluation shows that PULL consistently outperforms ten benchmark baselines across three different datasets and detects anomalous log messages with an F1-score of more than 0.99 even within imprecise failure time windows
CLIP-TSA: CLIP-Assisted Temporal Self-Attention for Weakly-Supervised Video Anomaly Detection
Video anomaly detection (VAD) -- commonly formulated as a multiple-instance
learning problem in a weakly-supervised manner due to its labor-intensive
nature -- is a challenging problem in video surveillance where the frames of
anomaly need to be localized in an untrimmed video. In this paper, we first
propose to utilize the ViT-encoded visual features from CLIP, in contrast with
the conventional C3D or I3D features in the domain, to efficiently extract
discriminative representations in the novel technique. We then model long- and
short-range temporal dependencies and nominate the snippets of interest by
leveraging our proposed Temporal Self-Attention (TSA). The ablation study
conducted on each component confirms its effectiveness in the problem, and the
extensive experiments show that our proposed CLIP-TSA outperforms the existing
state-of-the-art (SOTA) methods by a large margin on two commonly-used
benchmark datasets in the VAD problem (UCF-Crime and ShanghaiTech Campus). The
source code will be made publicly available upon acceptance.Comment: Under Submissio
A Survey on Unsupervised Anomaly Detection Algorithms for Industrial Images
In line with the development of Industry 4.0, surface defect
detection/anomaly detection becomes a topical subject in the industry field.
Improving efficiency as well as saving labor costs has steadily become a matter
of great concern in practice, where deep learning-based algorithms perform
better than traditional vision inspection methods in recent years. While
existing deep learning-based algorithms are biased towards supervised learning,
which not only necessitates a huge amount of labeled data and human labor, but
also brings about inefficiency and limitations. In contrast, recent research
shows that unsupervised learning has great potential in tackling the above
disadvantages for visual industrial anomaly detection. In this survey, we
summarize current challenges and provide a thorough overview of recently
proposed unsupervised algorithms for visual industrial anomaly detection
covering five categories, whose innovation points and frameworks are described
in detail. Meanwhile, publicly available datasets for industrial anomaly
detection are introduced. By comparing different classes of methods, the
advantages and disadvantages of anomaly detection algorithms are summarized.
Based on the current research framework, we point out the core issue that
remains to be resolved and provide further improvement directions. Meanwhile,
based on the latest technological trends, we offer insights into future
research directions. It is expected to assist both the research community and
industry in developing a broader and cross-domain perspective
- …