The Intriguing Relation Between Counterfactual Explanations and Adversarial Examples

The same method that creates adversarial examples (AEs) to fool image-classifiers can be used to generate counterfactual explanations (CEs) that explain algorithmic decisions. This observation has led researchers to consider CEs as AEs by another name. We argue that the relationship to the true label and the tolerance with respect to proximity are two properties that formally distinguish CEs and AEs. Based on these arguments, we introduce CEs, AEs, and related concepts mathematically in a common framework. Furthermore, we show connections between current methods for generating CEs and AEs, and estimate that the fields will merge more and more as the number of common use-cases grows

MineScratch: integração minetest-scratch para apoiar o ensino de programação

TCC(graduação) - Universidade Federal de Santa Catarina. Centro Tecnológico. Sistemas de Informação.O ensino de programação para crianças e jovens tem se tornado um assunto cada vez mais importante, entretanto muitas escolas ainda não oferecem esse tipo de oportunidade aos estudantes. Diversas iniciativas têm sido criadas no intuito de estimular e apoiar o ensino de programação. O Scratch é uma dessas ferramentas que permite o ensino de programação de forma visual. Por outro lado, jogos como Minecraft são muito atrativos para crianças e jovens. Minetest é um jogo com propósito similar, porém com código livre. Nesse sentido, a utilização de jogos pode ser um importante aliado no ensino desse conteúdo. Pensando nisso, o objetivo deste trabalho é apresentar uma ferramenta para auxiliar a construção de jogos educacionais para o ensino de programação, utilizando o apelo visual disponível pelo software Minetest e a facilidade de codificação do Scratch. O trabalho objetiva ainda a construção de um protótipo de jogo educacional, utilizando a biblioteca construída. Uma integração de integração entre Minetest e Scratch é modelada, desenvolvida, testada e avaliada. Os primeiros resultados da avaliação indicam a viabilidade de utilização da biblioteca

What does explainable AI explain?

Machine Learning (ML) models are increasingly used in industry, as well as in scientific research and social contexts. Unfortunately, ML models provide only partial solutions to real-world problems, focusing on predictive performance in static environments. Problem aspects beyond prediction, such as robustness in employment, knowledge generation in science, or providing recourse recommendations to end-users, cannot be directly tackled with ML models. Explainable Artificial Intelligence (XAI) aims to solve, or at least highlight, problem aspects beyond predictive performance through explanations. However, the field is still in its infancy, as fundamental questions such as “What are explanations?”, “What constitutes a good explanation?”, or “How relate explanation and understanding?” remain open. In this dissertation, I combine philosophical conceptual analysis and mathematical formalization to clarify a prerequisite of these difficult questions, namely what XAI explains: I point out that XAI explanations are either associative or causal and either aim to explain the ML model or the modeled phenomenon. The thesis is a collection of five individual research papers that all aim to clarify how different problems in XAI are related to these different “whats”. In Paper I, my co-authors and I illustrate how to construct XAI methods for inferring associational phenomenon relationships. Paper II directly relates to the first; we formally show how to quantify uncertainty of such scientific inferences for two XAI methods – partial dependence plots (PDP) and permutation feature importance (PFI). Paper III discusses the relationship between counterfactual explanations and adversarial examples; I argue that adversarial examples can be described as counterfactual explanations that alter the prediction but not the underlying target variable. In Paper IV, my co-authors and I argue that algorithmic recourse recommendations should help data-subjects improve their qualification rather than to game the predictor. In Paper V, we address general problems with model agnostic XAI methods and identify possible solutions

Test et évaluation de la robustesse de la couche fonctionnelle d'un robot autonome

La mise en oeuvre de systèmes autonomes nécessite le développement et l'utilisation d'architectures logicielles multi-couches qui soient adaptées. Typiquement, une couche fonctionnelle renferme des modules en charge de commander les éléments matériels du système et de fournir des services élémentaires. Pour être robuste, la couche fonctionnelle doit être dotée de mécanismes de protection vis-à-vis de requêtes erronées ou inopportunes issues de la couche supérieure. Nous présentons une méthodologie pour tester la robustesse de ces mécanismes. Nous définissons un cadre général pour évaluer la robustesse d'une couche fonctionnelle par la caractérisation de son comportement vis-à-vis de requêtes inopportunes. Nous proposons également un environnement de validation basé sur l'injection de fautes dans le logiciel de commande d'un robot simulé. Un grand nombre de cas de tests est généré automatiquement par la mutation d'une séquence de requêtes valides. Les statistiques descriptives des comportements en présence de requêtes inopportunes sont analysées afin d'évaluer la robustesse du système sous test. ABSTRACT : The implementation of autonomous systems requires the development and the using of multi-layer software architecture. Typically, a functional layer contains several modules that control the material of the system and provide elementary services. To be robust, the functional layer must be implemented with protection mechanisms with respect to erroneous or inopportune requests sent from the superior layer. We present a methodology for robustness testing these mechanisms. We define a general framework to evaluate the robustness of a functional layer by characterizing its behavior with respect to inappropriate requests. We also propose an validation environment based on fault injection in the control software of a simulated robot. A great number of test cases is generated automatically by the mutation of a sequence of valid requests. The descriptive statistics of the behaviors in the presence of inappropriate requests are analyzed in order to evaluate the robustness of the system under test

Context-aware task scheduling in distributed computing systems

These days, the popularity of technologies such as machine learning, augmented reality, and big data analytics is growing dramatically. This leads to a higher demand of computational power not only for IT professionals but also for ordinary device users who benefit from new applications. At the same time, the computational performance of end-user devices increases to meet the demands of these resource-hungry applications. As a result, there is a coexistence of a huge demand of computational power on the one side and a large pool of computational resources on the other side. Bringing these two sides together is the idea of computational resource sharing systems which allow applications to forward computationally intensive workload to remote resources. This technique is often used in cloud computing where customers can rent computational power. However, we argue that not only cloud resources can be used as offloading targets. Rather, idle CPU cycles from end-user administered devices at the edge of the network can be spontaneously leveraged as well. Edge devices, however, are not only heterogeneous in their hardware and software capabilities, they also do not provide any guarantees in terms of reliability or performance. Does it mean that either the applications that require further guarantees or the unpredictable resources need to be excluded from such a sharing system? In this thesis, we propose a solution to this problem by introducing the Tasklet system, our approach for a computational resource sharing system. The Tasklet system supports computation offloading to arbitrary types of devices, including stable cloud instances as well as unpredictable end-user owned edge resources. Therefore, the Tasklet system is structured into multiple layers. The lowest layer is a best-effort resource sharing system which provides lightweight task scheduling and execution. Here, best-effort means that in case of a failure, the task execution is dropped and that tasks are allocated to resources randomly. To provide execution guarantees such as a reliable or timely execution, we add a Quality of Computation (QoC) layer on top of the best-effort execution layer. The QoC layer enforces the guarantees for applications by using a context-aware task scheduler which monitors the available resources in the computing environment and performs the matchmaking between resources and tasks based on the current state of the system. As edge resources are controlled by individuals, we consider the fact that these users need to be able to decide with whom they want to share their resources and for which price. Thus, we add a social layer on top of the system that allows users to establish friendship connections which can then be leveraged for social-aware task allocation and accounting of shared computation

Resilience-Building Technologies: State of Knowledge -- ReSIST NoE Deliverable D12

This document is the first product of work package WP2, "Resilience-building and -scaling technologies", in the programme of jointly executed research (JER) of the ReSIST Network of Excellenc

A model-based approach for robustness testing

Abstract. Robustness testing is a part of the validation process which consists in testing the behavior of a system implementation under exceptional execution conditions in order to check if it still fulfills some robustness requirements. We propose a theoretical framework for modelbased robustness testing together with an implementation within the If validation environment. Robustness test cases are generated from both a (partial) operational specification and an abstract fault model. This generation technique is inspired from the ones used in (classical) conformance testing -already implemented in several tools. This framework is illustrated on a small example