Open-Category Classification by Adversarial Sample Generation

In real-world classification tasks, it is difficult to collect training samples from all possible categories of the environment. Therefore, when an instance of an unseen class appears in the prediction stage, a robust classifier should be able to tell that it is from an unseen class, instead of classifying it to be any known category. In this paper, adopting the idea of adversarial learning, we propose the ASG framework for open-category classification. ASG generates positive and negative samples of seen categories in the unsupervised manner via an adversarial learning strategy. With the generated samples, ASG then learns to tell seen from unseen in the supervised manner. Experiments performed on several datasets show the effectiveness of ASG.Comment: Published in IJCAI 201

Active Learning Methodology for Expert-Assisted Anomaly Detection in Mobile Communications

Due to the great complexity, heterogeneity, and variety of services, anomaly detection is becoming an increasingly important challenge in the operation of new generations of mobile communications. In many cases, the underlying relationships between the multiplicity of parameters and factors that can cause anomalous behavior are only determined by human expert knowledge. On the other hand, although automatic algorithms have a great capacity to process multiple sources of information, they are not always able to correctly signal such abnormalities. In this sense, this paper proposes the integration of both components in a framework based on Active Learning that enables enhanced performance in anomaly detection tasks. A series of tests have been conducted using an online anomaly detection algorithm comparing the proposed solution with a method based on the algorithm output alone. The obtained results demonstrate that a hybrid anomaly detection model that automates part of the process and includes the knowledge of an expert following the described methodology yields increased performance.This project is partially funded by the Junta de Andalucía through the UMA-CEIATECH-11 (DAMA-5G) project. It is also framed in the PENTA Excellence Project (P18-FR-4647) by the Consejería de Transformación Económica, Industria, Conocimiento y Universidades (Regional Ministry of Economic Transformation, Industry, Knowledge and Universities), and in part by the European Union–Next Generation EU within the Framework of the Project “Massive AI for the Open RadIo b5G/6G Network (MAORI)”. Partial funding for open access charge: Universidad de Málag

A study on labeling network hostile behavior with Intelligent Interactive tools

Labeling a real network dataset is specially expensive in computersecurity, as an expert has to ponder several factors before assigningeach label. This paper describes an interactive intelligent systemto support the task of identifying hostile behaviors in network logs.The RiskID application uses visualizations to graphically encodefeatures of network connections and promote visual comparison. Inthe background, two algorithms are used to actively organize con-nections and predict potential labels: a recommendation algorithmand a semi-supervised learning strategy. These algorithms togetherwith interactive adaptions to the user interface constitute a behaviorrecommendation. A study is carried out to analyze how the algo-rithms for recommendation and prediction influence the workflowof labeling a dataset. The results of a study with 16 participantsindicate that the behaviour recommendation significantly improvesthe quality of labels. Analyzing interaction patterns, we identify amore intuitive workflow used when behaviour recommendation isavailable.Fil: Guerra Torres, Jorge Luis. Universidad Nacional de Cuyo; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Mendoza; ArgentinaFil: Veas, Eduardo Enrique. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Mendoza; Argentina. Universidad Nacional de Cuyo; ArgentinaFil: Catania, Carlos Adrian. Universidad Nacional de Cuyo; Argentina2019 IEEE Symposium on Visualization for Cyber SecurityVancouverCanadáInstitute of Electrical and Electronics Engineer

Active Learning with Rationales for Identifying Operationally Significant Anomalies in Aviation

A major focus of the commercial aviation community is discovery of unknown safety events in flight operations data. Data-driven unsupervised anomaly detection methods are better at capturing unknown safety events compared to rule-based methods which only look for known violations. However, not all statistical anomalies that are discovered by these unsupervised anomaly detection methods are operationally significant (e.g., represent a safety concern). Subject Matter Experts (SMEs) have to spend significant time reviewing these statistical anomalies individually to identify a few operationally significant ones. In this paper we propose an active learning algorithm that incorporates SME feedback in the form of rationales to build a classifier that can distinguish between uninteresting and operationally significant anomalies. Experimental evaluation on real aviation data shows that our approach improves detection of operationally significant events by as much as 75% compared to the state-of-the-art. The learnt classifier also generalizes well to additional validation data sets

Sintel: A Machine Learning Framework to Extract Insights from Signals

The detection of anomalies in time series data is a critical task with many monitoring applications. Existing systems often fail to encompass an end-to-end detection process, to facilitate comparative analysis of various anomaly detection methods, or to incorporate human knowledge to refine output. This precludes current methods from being used in real-world settings by practitioners who are not ML experts. In this paper, we introduce Sintel, a machine learning framework for end-to-end time series tasks such as anomaly detection. The framework uses state-of-the-art approaches to support all steps of the anomaly detection process. Sintel logs the entire anomaly detection journey, providing detailed documentation of anomalies over time. It enables users to analyze signals, compare methods, and investigate anomalies through an interactive visualization tool, where they can annotate, modify, create, and remove events. Using these annotations, the framework leverages human knowledge to improve the anomaly detection pipeline. We demonstrate the usability, efficiency, and effectiveness of Sintel through a series of experiments on three public time series datasets, as well as one real-world use case involving spacecraft experts tasked with anomaly analysis tasks. Sintel's framework, code, and datasets are open-sourced at https://github.com/sintel-dev/.Comment: This work is accepted by ACM SIGMOD/PODS International Conference on Management of Data (SIGMOD 2022