3,476 research outputs found
Verbal Explanations for Deep Reinforcement Learning Neural Networks with Attention on Extracted Features
In recent years, there has been increasing interest in transparency in Deep Neural Networks. Most of the works on transparency have been done for image classification. In this paper, we report on work of transparency in Deep Reinforcement Learning Networks (DRLNs). Such networks have been extremely successful in learning action control in Atari games. In this paper, we focus on generating verbal (natural language) descriptions and explanations of deep reinforcement learning policies. Successful generation of verbal explanations would allow better understanding by people (e.g., users, debuggers) of the inner workings of DRLNs which could ultimately increase trust in these systems. We present a generation model which consists of three parts: an encoder on feature extraction, an attention structure on selecting features from the output of the encoder, and a decoder on generating the explanation in natural language. Four variants of the attention structure full attention, global attention, adaptive attention and object attention - are designed and compared. The adaptive attention structure performs the best among all the variants, even though the object attention structure is given additional information on object locations. Additionally, our experiment results showed that the proposed encoder outperforms two baseline encoders (Resnet and VGG) on the capability of distinguishing the game state images
Deep Learning, transparency and trust in Human Robot Teamwork
For Autonomous AI systems to be accepted and trusted, the users should be able to understand the reasoning process of the system (i.e., the system should be transparent). Robotics presents unique programming difficulties in that systems need to map from complicated sensor inputs such as camera feeds and laser scans to outputs such as joint angles and velocities. Advances in Deep Neural Networks are now making it possible to replace laborious handcrafted features and control code by learning control policies directly from high dimensional sensor inputs. Because Atari games, where these capabilities were first demonstrated, replicate the robotics problem they are ideal for investigating how humans might come to understand and interact with agents who have not been explicitly programmed. We present computational and human results for making DRLN more transparent using object saliency visualizations of internal states and test the effectiveness of expressing saliency through teleological verbal explanations
Unmasking Clever Hans Predictors and Assessing What Machines Really Learn
Current learning machines have successfully solved hard application problems,
reaching high accuracy and displaying seemingly "intelligent" behavior. Here we
apply recent techniques for explaining decisions of state-of-the-art learning
machines and analyze various tasks from computer vision and arcade games. This
showcases a spectrum of problem-solving behaviors ranging from naive and
short-sighted, to well-informed and strategic. We observe that standard
performance evaluation metrics can be oblivious to distinguishing these diverse
problem solving behaviors. Furthermore, we propose our semi-automated Spectral
Relevance Analysis that provides a practically effective way of characterizing
and validating the behavior of nonlinear learning machines. This helps to
assess whether a learned model indeed delivers reliably for the problem that it
was conceived for. Furthermore, our work intends to add a voice of caution to
the ongoing excitement about machine intelligence and pledges to evaluate and
judge some of these recent successes in a more nuanced manner.Comment: Accepted for publication in Nature Communication
Explaining Aha! moments in artificial agents through IKE-XAI: Implicit Knowledge Extraction for eXplainable AI
During the learning process, a child develops a mental representation of the task he or she is learning.
A Machine Learning algorithm develops also a latent representation of the task it learns. We investigate
the development of the knowledge construction of an artificial agent through the analysis of its
behavior, i.e., its sequences of moves while learning to perform the Tower of HanoĂŻ (TOH) task. The TOH
is a well-known task in experimental contexts to study the problem-solving processes and one of the
fundamental processes of children’s knowledge construction about their world. We position ourselves
in the field of explainable reinforcement learning for developmental robotics, at the crossroads of
cognitive modeling and explainable AI. Our main contribution proposes a 3-step methodology named
Implicit Knowledge Extraction with eXplainable Artificial Intelligence (IKE-XAI) to extract the implicit
knowledge, in form of an automaton, encoded by an artificial agent during its learning. We showcase
this technique to solve and explain the TOH task when researchers have only access to moves that
represent observational behavior as in human–machine interaction. Therefore, to extract the agent
acquired knowledge at different stages of its training, our approach combines: first, a Q-learning
agent that learns to perform the TOH task; second, a trained recurrent neural network that encodes
an implicit representation of the TOH task; and third, an XAI process using a post-hoc implicit rule
extraction algorithm to extract finite state automata. We propose using graph representations as visual
and explicit explanations of the behavior of the Q-learning agent. Our experiments show that the IKEXAI
approach helps understanding the development of the Q-learning agent behavior by providing
a global explanation of its knowledge evolution during learning. IKE-XAI also allows researchers to
identify the agent’s Aha! moment by determining from what moment the knowledge representation
stabilizes and the agent no longer learns.Region BretagneEuropean Union via the FEDER programSpanish Government Juan de la Cierva Incorporacion - MCIN/AEI IJC2019-039152-IGoogle Research Scholar Gran
Building Machines That Learn and Think Like People
Recent progress in artificial intelligence (AI) has renewed interest in
building systems that learn and think like people. Many advances have come from
using deep neural networks trained end-to-end in tasks such as object
recognition, video games, and board games, achieving performance that equals or
even beats humans in some respects. Despite their biological inspiration and
performance achievements, these systems differ from human intelligence in
crucial ways. We review progress in cognitive science suggesting that truly
human-like learning and thinking machines will have to reach beyond current
engineering trends in both what they learn, and how they learn it.
Specifically, we argue that these machines should (a) build causal models of
the world that support explanation and understanding, rather than merely
solving pattern recognition problems; (b) ground learning in intuitive theories
of physics and psychology, to support and enrich the knowledge that is learned;
and (c) harness compositionality and learning-to-learn to rapidly acquire and
generalize knowledge to new tasks and situations. We suggest concrete
challenges and promising routes towards these goals that can combine the
strengths of recent neural network advances with more structured cognitive
models.Comment: In press at Behavioral and Brain Sciences. Open call for commentary
proposals (until Nov. 22, 2016).
https://www.cambridge.org/core/journals/behavioral-and-brain-sciences/information/calls-for-commentary/open-calls-for-commentar
Creativity and the Brain
Neurocognitive approach to higher cognitive functions that bridges the gap between psychological and neural level of description is introduced. Relevant facts about the brain, working memory and representation of symbols in the brain are summarized. Putative brain processes responsible for problem solving, intuition, skill learning and automatization are described. The role of non-dominant brain hemisphere in solving problems requiring insight is conjectured. Two factors seem to be essential for creativity: imagination constrained by experience, and filtering that selects most interesting solutions. Experiments with paired words association are analyzed in details and evidence for stochastic resonance effects is found. Brain activity in the process of invention of novel words is proposed as the simplest way to understand creativity using experimental and computational means. Perspectives on computational models of creativity are discussed
- …