ADDP: Anomaly Detection based on Denoising Pretraining

    Acquiring labels in anomaly detection tasks is expensive and challenging. Therefore, as an effective way to improve efficiency, pretraining is widely used in anomaly detection models， which enriches the model's representation capabilities, thereby enhancing both performance and efficiency in anomaly detection. In most pretraining methods, the decoder is typically randomly initialized. Drawing inspiration from the diffusion model, this paper proposed to use denoising as a task to pretrain the decoder in anomaly detection, which is trained to reconstruct the original noise-free input. Denoising requires the model to learn the structure, patterns, and related features of the data, particularly when training samples are limited. This paper explored two approaches on anomaly detection: simultaneous denoising pretraining for encoder and decoder, denoising pretraining for only decoder. Experimental results demonstrate the effectiveness of this method on improving model’s performance. Particularly, when the number of samples is limited, the improvement is more pronounced

ADDP: Anomaly Detection based on Denoising Pretraining

    Acquiring labels in anomaly detection tasks is expensive and challenging. Therefore, as an effective way to improve efficiency, pretraining is widely used in anomaly detection models， which enriches the model\u27s representation capabilities, thereby enhancing both performance and efficiency in anomaly detection. In most pretraining methods, the decoder is typically randomly initialized. Drawing inspiration from the diffusion model, this paper proposed to use denoising as a task to pretrain the decoder in anomaly detection, which is trained to reconstruct the original noise-free input. Denoising requires the model to learn the structure, patterns, and related features of the data, particularly when training samples are limited. This paper explored two approaches on anomaly detection: simultaneous denoising pretraining for encoder and decoder, denoising pretraining for only decoder. Experimental results demonstrate the effectiveness of this method on improving model’s performance. Particularly, when the number of samples is limited, the improvement is more pronounced

Anomaly Awareness

We present a new Machine Learning algorithm called Anomaly Awareness. By making our algorithm aware of the presence of a range of different anomalies, we improve its capability to detect anomalous events, even those it had not been exposed to. As an example of use, we apply this method to searches for new phenomena in the Large Hadron Collider. In particular, we analyze events with boosted jets where new physics could be hiding.Comment: 8 pages, 11 figure

Искусственный интеллект в медицине: современное состояние и основные направления развития интеллектуальной диагностики

The main difference between artificial intelligence (AI) systems and simple automated algorithms is the ability to learn, synthesize and conclude. The AI system is trained on a set of examples, including pictures, characteristics of patients with a certain disease, then it allows to generalize a lot of such examples and get some general functional dependence, which brings in line the patient data and a certain diagnosis. The system can be named intelligent if this synthetizing ability is realized. Although the AI systems are now becoming more understood and accepted by doctors, a deeper understanding of «how it works» is needed. The article provides a detailed review of the application of methods and models of artificial intelligence in the diagnostics of cancer based on the of multimodal instrumental data. The basic concepts of artificial intelligence and directions of its development are presented. From the point of view of data processing, the stages of development of AI systems are identical. The stages of intellectual processing of diagnostic data are considered in the paper. They include the acquisition and use of training databases of oncological diseases, pre-processing of images, segmentation to highlight the studied objects of diagnosis and classification of these objects to determine whether they are malignant or benign. One of the problems limiting the acceptance of AI systems development by the medical community is the imperfection of the explainability of the results obtained by intelligent systems. Authors pay attention to importance of the development of so-called explanatory intelligence, because its absence currently significantly inhibits the introduction and use of intelligent diagnostic systems in medicine. In addition, the purpose of the article is a way to develop the interaction between a radiologists and data scientists.Главное отличие систем искусственного интеллекта (ИИ) от простых автоматизированных алгоритмов заключается в способности к обучению, обобщению и выводу. Система ИИ обучается на множестве примеров, включая снимки, характеристики пациентов с определенным заболеванием, далее она позволяет обобщить множество таких примеров и получить некоторую общую функциональную зависимость, которая приводит в соответствие данные о пациенте и определенный диагноз. Интеллектуальной система становится при реализации этой обобщающей способности. Несмотря на то, что в настоящее время тематика ИИ становится более понимаемой и принимаемой врачами, необходимо более глубокое понимание «как это работает». В статье приводится детальный обзор применения методов и моделей искусственного интеллекта в диагностике онкологических заболеваний на основе данных мультимодальной лучевой диагностики. Даны основные понятия искусственного интеллекта и направления его использования. С точки зрения обработки данных этапы разработки систем ИИ идентичны. В статье рассмотрены этапы интеллектуальной обработки диагностических данных, которые включают создание и использование обучающих баз данных онкологических заболеваний, предварительную обработку снимков, сегментацию изображений для выделения исследуемых объектов диагностики и классификацию этих объектов для определения, являются ли они злокачественными или доброкачественными. Одной из проблем, ограничивающих принятие развития систем ИИ медицинским сообществом, является несовершенство объяснимости результатов, получаемых при помощи интеллектуальных систем. В статье затронуты важные вопросы разработки объяснительного интеллекта, отсутствие которого в настоящее время существенно тормозит внедрение и использование интеллектуальных систем диагностики в медицине. Кроме того, цель статьи — путь к развитию взаимодействия между врачом и специалистом по искусственному интеллекту

Positive Difference Distribution for Image Outlier Detection using Normalizing Flows and Contrastive Data

Detecting test data deviating from training data is a central problem for safe and robust machine learning. Likelihoods learned by a generative model, e.g., a normalizing flow via standard log-likelihood training, perform poorly as an outlier score. We propose to use an unlabelled auxiliary dataset and a probabilistic outlier score for outlier detection. We use a self-supervised feature extractor trained on the auxiliary dataset and train a normalizing flow on the extracted features by maximizing the likelihood on in-distribution data and minimizing the likelihood on the contrastive dataset. We show that this is equivalent to learning the normalized positive difference between the in-distribution and the contrastive feature density. We conduct experiments on benchmark datasets and compare to the likelihood, the likelihood ratio and state-of-the-art anomaly detection methods

Spatiotemporal anomaly detection: streaming architecture and algorithms

Includes bibliographical references.2020 Summer.Anomaly detection is the science of identifying one or more rare or unexplainable samples or events in a dataset or data stream. The field of anomaly detection has been extensively studied by mathematicians, statisticians, economists, engineers, and computer scientists. One open research question remains the design of distributed cloud-based architectures and algorithms that can accurately identify anomalies in previously unseen, unlabeled streaming, multivariate spatiotemporal data. With streaming data, time is of the essence, and insights are perishable. Real-world streaming spatiotemporal data originate from many sources, including mobile phones, supervisory control and data acquisition enabled (SCADA) devices, the internet-of-things (IoT), distributed sensor networks, and social media. Baseline experiments are performed on four (4) non-streaming, static anomaly detection multivariate datasets using unsupervised offline traditional machine learning (TML), and unsupervised neural network techniques. Multiple architectures, including autoencoders, generative adversarial networks, convolutional networks, and recurrent networks, are adapted for experimentation. Extensive experimentation demonstrates that neural networks produce superior detection accuracy over TML techniques. These same neural network architectures can be extended to process unlabeled spatiotemporal streaming using online learning. Space and time relationships are further exploited to provide additional insights and increased anomaly detection accuracy. A novel domain-independent architecture and set of algorithms called the Spatiotemporal Anomaly Detection Environment (STADE) is formulated. STADE is based on federated learning architecture. STADE streaming algorithms are based on a geographically unique, persistently executing neural networks using online stochastic gradient descent (SGD). STADE is designed to be pluggable, meaning that alternative algorithms may be substituted or combined to form an ensemble. STADE incorporates a Stream Anomaly Detector (SAD) and a Federated Anomaly Detector (FAD). The SAD executes at multiple locations on streaming data, while the FAD executes at a single server and identifies global patterns and relationships among the site anomalies. Each STADE site streams anomaly scores to the centralized FAD server for further spatiotemporal dependency analysis and logging. The FAD is based on recent advances in DNN-based federated learning. A STADE testbed is implemented to facilitate globally distributed experimentation using low-cost, commercial cloud infrastructure provided by Microsoft™. STADE testbed sites are situated in the cloud within each continent: Africa, Asia, Australia, Europe, North America, and South America. Communication occurs over the commercial internet. Three STADE case studies are investigated. The first case study processes commercial air traffic flows, the second case study processes global earthquake measurements, and the third case study processes social media (i.e., Twitter™) feeds. These case studies confirm that STADE is a viable architecture for the near real-time identification of anomalies in streaming data originating from (possibly) computationally disadvantaged, geographically dispersed sites. Moreover, the addition of the FAD provides enhanced anomaly detection capability. Since STADE is domain-independent, these findings can be easily extended to additional application domains and use cases