Causal Confusion in Imitation Learning

Behavioral cloning reduces policy learning to supervised learning by training a discriminative model to predict expert actions given observations. Such discriminative models are non-causal: the training procedure is unaware of the causal structure of the interaction between the expert and the environment. We point out that ignoring causality is particularly damaging because of the distributional shift in imitation learning. In particular, it leads to a counter-intuitive "causal misidentification" phenomenon: access to more information can yield worse performance. We investigate how this problem arises, and propose a solution to combat it through targeted interventions---either environment interaction or expert queries---to determine the correct causal model. We show that causal misidentification occurs in several benchmark control domains as well as realistic driving settings, and validate our solution against DAgger and other baselines and ablations.Comment: Published at NeurIPS 2019 9 pages, plus references and appendice

Generative Speech Recognition Error Correction with Large Language Models and Task-Activating Prompting

We explore the ability of large language models (LLMs) to act as speech recognition post-processors that perform rescoring and error correction. Our first focus is on instruction prompting to let LLMs perform these task without fine-tuning, for which we evaluate different prompting schemes, both zero- and few-shot in-context learning, and a novel task activation prompting method that combines causal instructions and demonstration to increase its context windows. Next, we show that rescoring only by in-context learning with frozen LLMs achieves results that are competitive with rescoring by domain-tuned LMs, using a pretrained first-pass recognition system and rescoring output on two out-of-domain tasks (ATIS and WSJ). By combining prompting techniques with fine-tuning we achieve error rates below the N-best oracle level, showcasing the generalization power of the LLMs.Comment: Accepted to IEEE Automatic Speech Recognition and Understanding (ASRU) 2023. 8 pages. 2nd version revised from Sep 29th's versio

The interaction of social and perceivable causal factors in shaping ‘over-imitation’

Over-imitation has become a well-documented phenomenon. However there is evidence that both social and visible, physically causal factors can influence the occurrence of over-imitation in children. Here we explore the interplay between these two factors, manipulating both task opacity and social information. Four- to 7-year-old children were given either a causally opaque or transparent box, before which they experienced either (1) a condition where they witnessed a taught, knowledgeable person demonstrate an inefficient method and an untaught model demonstrate a more efficient method; or (2) a baseline condition where they witnessed efficient and inefficient methods performed by two untaught models. Results showed that the level of imitation increased with greater task opacity and when children received social information about knowledgeability consequent on teaching, but only for 6- to 7-year-olds. The findings show that children are selectively attuned to both causal and social factors when learning new cultural knowledge

From motor babbling to hierarchical learning by imitation: a robot developmental pathway

How does an individual use the knowledge acquired through self exploration as a manipulable model through which to understand others and benefit from their knowledge? How can developmental and social learning be combined for their mutual benefit? In this paper we review a hierarchical architecture (HAMMER) which allows a principled way for combining knowledge through exploration and knowledge from others, through the creation and use of multiple inverse and forward models. We describe how Bayesian Belief Networks can be used to learn the association between a robot’s motor commands and sensory consequences (forward models), and how the inverse association can be used for imitation. Inverse models created through self exploration, as well as those from observing others can coexist and compete in a principled unified framework, that utilises the simulation theory of mind approach to mentally rehearse and understand the actions of others

High-Tech Tools for Teaching Physics: the Physics Education Technology Project

This article appeared in the Journal of Online Teaching and Learning September 15, 2006.This paper introduces a new suite of computer simulations from the Physics Education Technology (PhET) project, identifies features of these educational tools, and demonstrates their utility. We compare the use of PhET simulations to the use of more traditional educational resources in lecture, laboratory, recitation and informal settings of introductory college physics. In each case we demonstrate that simulations are as productive, or more productive, for developing student conceptual understanding as real equipment, reading resources, or chalk-talk lectures. We further identify six key characteristic features of these simulations that begin to delineate why these are productive tools. The simulations: support an interactive approach, employ dynamic feedback, follow a constructivist approach, provide a creative workplace, make explicit otherwise inaccessible models or phenomena, and constrain students productively

Causal Induction from Continuous Event Streams: Evidence for Delay-Induced Attribution Shifts

Contemporary theories of Human Causal Induction assume that causal knowledge is inferred from observable contingencies. While this assumption is well supported by empirical results, it fails to consider an important problem-solving aspect of causal induction in real time: In the absence of well structured learning trials, it is not clear whether the effect of interest occurred because of the cause under investigation, or on its own accord. Attributing the effect to either the cause of interest or alternative background causes is an important precursor to induction. We present a new paradigm based on the presentation of continuous event streams, and use it to test the Attribution-Shift Hypothesis (Shanks & Dickinson, 1987), according to which temporal delays sever the attributional link between cause and effect. Delays generally impaired attribution to the candidate, and increased attribution to the constant background of alternative causes. In line with earlier research (Buehner & May, 2002, 2003, 2004) prior knowledge and experience mediated this effect. Pre-exposure to a causally ineffective background context was found to facilitate the discovery of delayed causal relationships by reducing the tendency for attributional shifts to occur. However, longer exposure to a delayed causal relationship did not improve discovery. This complex pattern of results is problematic for associative learning theories, but supports the Attribution-Shift Hypothesi