Anomaly Detection and Explanation Discovery on Event Streams

International audienceAs enterprise information systems are collecting event streams from various sources, the ability of a system to automatically detect anomalous events and further provide human readable explanations is of paramount importance. In this position paper, we argue for the need of a new type of data stream analytics that can address anomaly detection and explanation discovery in a single, integrated system, which not only offers increased business intelligence, but also opens up opportunities for improved solutions. In particular , we propose a two-pass approach to building such a system, highlight the challenges, and offer initial directions for solutions

Lightweight Automated Feature Monitoring for Data Streams

Monitoring the behavior of automated real-time stream processing systems has become one of the most relevant problems in real world applications. Such systems have grown in complexity relying heavily on high dimensional input data, and data hungry Machine Learning (ML) algorithms. We propose a flexible system, Feature Monitoring (FM), that detects data drifts in such data sets, with a small and constant memory footprint and a small computational cost in streaming applications. The method is based on a multi-variate statistical test and is data driven by design (full reference distributions are estimated from the data). It monitors all features that are used by the system, while providing an interpretable features ranking whenever an alarm occurs (to aid in root cause analysis). The computational and memory lightness of the system results from the use of Exponential Moving Histograms. In our experimental study, we analyze the system's behavior with its parameters and, more importantly, show examples where it detects problems that are not directly related to a single feature. This illustrates how FM eliminates the need to add custom signals to detect specific types of problems and that monitoring the available space of features is often enough.Comment: 10 pages, 5 figures. AutoML, KDD22, August 14-17, 2022, Washington, DC, U

Anomaly Detection for Industrial Big Data

As the Industrial Internet of Things (IIoT) grows, systems are increasingly being monitored by arrays of sensors returning time-series data at ever-increasing 'volume, velocity and variety' (i.e. Industrial Big Data). An obvious use for these data is real-time systems condition monitoring and prognostic time to failure analysis (remaining useful life, RUL). (e.g. See white papers by Senseye.io, and output of the NASA Prognostics Center of Excellence (PCoE).) However, as noted by Agrawal and Choudhary 'Our ability to collect "big data" has greatly surpassed our capability to analyze it, underscoring the emergence of the fourth paradigm of science, which is data-driven discovery.' In order to fully utilize the potential of Industrial Big Data we need data-driven techniques that operate at scales that process models cannot. Here we present a prototype technique for data-driven anomaly detection to operate at industrial scale. The method generalizes to application with almost any multivariate dataset based on independent ordinations of repeated (bootstrapped) partitions of the dataset and inspection of the joint distribution of ordinal distances.Comment: 9 pages; 11 figure

Hoeffding Tree Algorithms for Anomaly Detection in Streaming Datasets: A Survey

This survey aims to deliver an extensive and well-constructed overview of using machine learning for the problem of detecting anomalies in streaming datasets. The objective is to provide the effectiveness of using Hoeffding Trees as a machine learning algorithm solution for the problem of detecting anomalies in streaming cyber datasets. In this survey we categorize the existing research works of Hoeffding Trees which can be feasible for this type of study into the following: surveying distributed Hoeffding Trees, surveying ensembles of Hoeffding Trees and surveying existing techniques using Hoeffding Trees for anomaly detection. These categories are referred to as compositions within this paper and were selected based on their relation to streaming data and the flexibility of their techniques for use within different domains of streaming data. We discuss the relevance of how combining the techniques of the proposed research works within these compositions can be used to address the anomaly detection problem in streaming cyber datasets. The goal is to show how a combination of techniques from different compositions can solve a prominent problem, anomaly detection

Demand pattern analysis of taxi trip data for anomalies detection and explanation

Novi Zakon o obveznim odnosima promijenio je naziv instituta bankarske garancije u bankarsko jamstvo i pojam tog instituta izložen u čl. 1039. st. 1. i 2. (tako da se sada pod nazivom bankarskog jamstva pojavljuje samostalna bankarska garancija), dok ostale odredbe ranijeg ZOO-a sadržajno nisu promijenjene. Bankovna garancija jeste samostalna obveza banke garanta koja je akcesorna obveza jamca. Banka garant ne osigurava ispunjenje obveze glavnog dužnika, naprotiv, obvezuje se korisniku garancije nadoknaditi štetu, odnosno izvršiti obvezu koju u ugovorenom roku nije izvršio glavni dužnik. U radu izlažem pitanja u svezi s oblikom i vrstama garancije, kvalifikacijom i nastankom bančine obveze prema korisniku, pretpostavkama isplate, prenosivošću i potvrdom garancije kao i njihovoj zlouporabi.The new Law of mandatory relations has changed the name of the bank warranty to bank assurance and complete connotation is represented in article 1039. in section 1 and 2. (according to which, under the name of bank warranty is independent bank assurance) while other provisions from the Law of mandatory relations have not been significantly contextually changed. Bank warranty is independent obligation of the warrant bank, which is accessory obligation of the guarantor. Warrant bank does not assure implementation of the main debtor’s obligation, but it commits to compensate potential detriment towards the warranty user, in the other words, implement the obligation which has not been realized by the main debtor in specified time period