Explainable and Advisable Learning for Self-driving Vehicles

Deep neural perception and control networks are likely to be a key component of self-driving vehicles. These models need to be explainable - they should provide easy-to-interpret rationales for their behavior - so that passengers, insurance companies, law enforcement, developers, etc., can understand what triggered a particular behavior. Explanations may be triggered by the neural controller, namely introspective explanations, or informed by the neural controller's output, namely rationalizations. Our work has focused on the challenge of generating introspective explanations of deep models for self-driving vehicles. In Chapter 3, we begin by exploring the use of visual explanations. These explanations take the form of real-time highlighted regions of an image that causally influence the network's output (steering control). In the first stage, we use a visual attention model to train a convolution network end-to-end from images to steering angle. The attention model highlights image regions that potentially influence the network's output. Some of these are true influences, but some are spurious. We then apply a causal filtering step to determine which input regions actually influence the output. This produces more succinct visual explanations and more accurately exposes the network's behavior. In Chapter 4, we add an attention-based video-to-text model to produce textual explanations of model actions, e.g. "the car slows down because the road is wet". The attention maps of controller and explanation model are aligned so that explanations are grounded in the parts of the scene that mattered to the controller. We explore two approaches to attention alignment, strong- and weak-alignment. These explainable systems represent an externalization of tacit knowledge. The network's opaque reasoning is simplified to a situation-specific dependence on a visible object in the image. This makes them brittle and potentially unsafe in situations that do not match training data. In Chapter 5, we propose to address this issue by augmenting training data with natural language advice from a human. Advice includes guidance about what to do and where to attend. We present the first step toward advice-giving, where we train an end-to-end vehicle controller that accepts advice. The controller adapts the way it attends to the scene (visual attention) and the control (steering and speed). Further, in Chapter 6, we propose a new approach that learns vehicle control with the help of long-term (global) human advice. Specifically, our system learns to summarize its visual observations in natural language, predict an appropriate action response (e.g. "I see a pedestrian crossing, so I stop"), and predict the controls, accordingly

A visual exploration workflow as enabler for the exploitation of Linked Open Data

Abstract. Semantically annotating and interlinking Open Data results in Linked Open Data which concisely and unambiguously describes a knowledge domain. However, the uptake of the Linked Data depends on its usefulness to non-Semantic Web experts. Failing to support data consumers to understand the added-value of Linked Data and possible exploitation opportunities could inhibit its diffusion. In this paper, we propose an interactive visual workflow for discovering and ex-ploring Linked Open Data. We implemented the workflow considering academic library metadata and carried out a qualitative evaluation. We assessed the work-flow’s potential impact on data consumers which bridges the offer: published Linked Open Data; and the demand as requests for: (i) higher quality data; and (ii) more applications that re-use data. More than 70 % of the 34 test users agreed that the workflow fulfills its goal: it facilitates non-Semantic Web experts to un-derstand the potential of Linked Open Data.

DRLViz: Understanding Decisions and Memory in Deep Reinforcement Learning

We present DRLViz, a visual analytics interface to interpret the internal memory of an agent (e.g. a robot) trained using deep reinforcement learning. This memory is composed of large temporal vectors updated when the agent moves in an environment and is not trivial to understand due to the number of dimensions, dependencies to past vectors, spatial/temporal correlations, and co-correlation between dimensions. It is often referred to as a black box as only inputs (images) and outputs (actions) are intelligible for humans. Using DRLViz, experts are assisted to interpret decisions using memory reduction interactions, and to investigate the role of parts of the memory when errors have been made (e.g. wrong direction). We report on DRLViz applied in the context of video games simulators (ViZDoom) for a navigation scenario with item gathering tasks. We also report on experts evaluation using DRLViz, and applicability of DRLViz to other scenarios and navigation problems beyond simulation games, as well as its contribution to black box models interpretability and explainability in the field of visual analytics

Flow-based Intrinsic Curiosity Module

In this paper, we focus on a prediction-based novelty estimation strategy upon the deep reinforcement learning (DRL) framework, and present a flow-based intrinsic curiosity module (FICM) to exploit the prediction errors from optical flow estimation as exploration bonuses. We propose the concept of leveraging motion features captured between consecutive observations to evaluate the novelty of observations in an environment. FICM encourages a DRL agent to explore observations with unfamiliar motion features, and requires only two consecutive frames to obtain sufficient information when estimating the novelty. We evaluate our method and compare it with a number of existing methods on multiple benchmark environments, including Atari games, Super Mario Bros., and ViZDoom. We demonstrate that FICM is favorable to tasks or environments featuring moving objects, which allow FICM to utilize the motion features between consecutive observations. We further ablatively analyze the encoding efficiency of FICM, and discuss its applicable domains comprehensively.Comment: The SOLE copyright holder is IJCAI (International Joint Conferences on Artificial Intelligence), all rights reserved. The link is provided as follows: https://www.ijcai.org/Proceedings/2020/28

From Social Simulation to Integrative System Design

As the recent financial crisis showed, today there is a strong need to gain "ecological perspective" of all relevant interactions in socio-economic-techno-environmental systems. For this, we suggested to set-up a network of Centers for integrative systems design, which shall be able to run all potentially relevant scenarios, identify causality chains, explore feedback and cascading effects for a number of model variants, and determine the reliability of their implications (given the validity of the underlying models). They will be able to detect possible negative side effect of policy decisions, before they occur. The Centers belonging to this network of Integrative Systems Design Centers would be focused on a particular field, but they would be part of an attempt to eventually cover all relevant areas of society and economy and integrate them within a "Living Earth Simulator". The results of all research activities of such Centers would be turned into informative input for political Decision Arenas. For example, Crisis Observatories (for financial instabilities, shortages of resources, environmental change, conflict, spreading of diseases, etc.) would be connected with such Decision Arenas for the purpose of visualization, in order to make complex interdependencies understandable to scientists, decision-makers, and the general public.Comment: 34 pages, Visioneer White Paper, see http://www.visioneer.ethz.c

Visual analytics for supply network management: system design and evaluation

We propose a visual analytic system to augment and enhance decision-making processes of supply chain managers. Several design requirements drive the development of our integrated architecture and lead to three primary capabilities of our system prototype. First, a visual analytic system must integrate various relevant views and perspectives that highlight different structural aspects of a supply network. Second, the system must deliver required information on-demand and update the visual representation via user-initiated interactions. Third, the system must provide both descriptive and predictive analytic functions for managers to gain contingency intelligence. Based on these capabilities we implement an interactive web-based visual analytic system. Our system enables managers to interactively apply visual encodings based on different node and edge attributes to facilitate mental map matching between abstract attributes and visual elements. Grounded in cognitive fit theory, we demonstrate that an interactive visual system that dynamically adjusts visual representations to the decision environment can significantly enhance decision-making processes in a supply network setting. We conduct multi-stage evaluation sessions with prototypical users that collectively confirm the value of our system. Our results indicate a positive reaction to our system. We conclude with implications and future research opportunities.The authors would like to thank the participants of the 2015 Businessvis Workshop at IEEE VIS, Prof. Benoit Montreuil, and Dr. Driss Hakimi for their valuable feedback on an earlier version of the software; Prof. Manpreet Hora for assisting with and Georgia Tech graduate students for participating in the evaluation sessions; and the two anonymous reviewers for their detailed comments and suggestions. The study was in part supported by the Tennenbaum Institute at Georgia Tech Award # K9305. (K9305 - Tennenbaum Institute at Georgia Tech Award)Accepted manuscrip