Predicting Motivations of Actions by Leveraging Text

Understanding human actions is a key problem in computer vision. However, recognizing actions is only the first step of understanding what a person is doing. In this paper, we introduce the problem of predicting why a person has performed an action in images. This problem has many applications in human activity understanding, such as anticipating or explaining an action. To study this problem, we introduce a new dataset of people performing actions annotated with likely motivations. However, the information in an image alone may not be sufficient to automatically solve this task. Since humans can rely on their lifetime of experiences to infer motivation, we propose to give computer vision systems access to some of these experiences by using recently developed natural language models to mine knowledge stored in massive amounts of text. While we are still far away from fully understanding motivation, our results suggest that transferring knowledge from language into vision can help machines understand why people in images might be performing an action.Comment: CVPR 201

Interactive Robot Learning of Gestures, Language and Affordances

A growing field in robotics and Artificial Intelligence (AI) research is human-robot collaboration, whose target is to enable effective teamwork between humans and robots. However, in many situations human teams are still superior to human-robot teams, primarily because human teams can easily agree on a common goal with language, and the individual members observe each other effectively, leveraging their shared motor repertoire and sensorimotor resources. This paper shows that for cognitive robots it is possible, and indeed fruitful, to combine knowledge acquired from interacting with elements of the environment (affordance exploration) with the probabilistic observation of another agent's actions. We propose a model that unites (i) learning robot affordances and word descriptions with (ii) statistical recognition of human gestures with vision sensors. We discuss theoretical motivations, possible implementations, and we show initial results which highlight that, after having acquired knowledge of its surrounding environment, a humanoid robot can generalize this knowledge to the case when it observes another agent (human partner) performing the same motor actions previously executed during training.Comment: code available at https://github.com/gsaponaro/glu-gesture

Bibliography of computational models of rat spatial behavior

A bibliography of computational models of rat spatial behavior

Enactivism and ethnomethodological conversation analysis as tools for expanding Universal Design for Learning: the case of visually impaired mathematics students

Blind and visually impaired mathematics students must rely on accessible materials such as tactile diagrams to learn mathematics. However, these compensatory materials are frequently found to offer students inferior opportunities for engaging in mathematical practice and do not allow sensorily heterogenous students to collaborate. Such prevailing problems of access and interaction are central concerns of Universal Design for Learning (UDL), an engineering paradigm for inclusive participation in cultural praxis like mathematics. Rather than directly adapt existing artifacts for broader usage, UDL process begins by interrogating the praxis these artifacts serve and then radically re-imagining tools and ecologies to optimize usability for all learners. We argue for the utility of two additional frameworks to enhance UDL efforts: (a) enactivism, a cognitive-sciences view of learning, knowing, and reasoning as modal activity; and (b) ethnomethodological conversation analysis (EMCA), which investigates participants’ multimodal methods for coordinating action and meaning. Combined, these approaches help frame the design and evaluation of opportunities for heterogeneous students to learn mathematics collaboratively in inclusive classrooms by coordinating perceptuo-motor solutions to joint manipulation problems. We contextualize the thesis with a proposal for a pluralist design for proportions, in which a pair of students jointly operate an interactive technological device

Intrinsic Motivation Systems for Autonomous Mental Development

Exploratory activities seem to be intrinsically rewarding for children and crucial for their cognitive development. Can a machine be endowed with such an intrinsic motivation system? This is the question we study in this paper, presenting a number of computational systems that try to capture this drive towards novel or curious situations. After discussing related research coming from developmental psychology, neuroscience, developmental robotics, and active learning, this paper presents the mechanism of Intelligent Adaptive Curiosity, an intrinsic motivation system which pushes a robot towards situations in which it maximizes its learning progress. This drive makes the robot focus on situations which are neither too predictable nor too unpredictable, thus permitting autonomous mental development.The complexity of the robot’s activities autonomously increases and complex developmental sequences self-organize without being constructed in a supervised manner. Two experiments are presented illustrating the stage-like organization emerging with this mechanism. In one of them, a physical robot is placed on a baby play mat with objects that it can learn to manipulate. Experimental results show that the robot first spends time in situations which are easy to learn, then shifts its attention progressively to situations of increasing difficulty, avoiding situations in which nothing can be learned. Finally, these various results are discussed in relation to more complex forms of behavioral organization and data coming from developmental psychology. Key words: Active learning, autonomy, behavior, complexity, curiosity, development, developmental trajectory, epigenetic robotics, intrinsic motivation, learning, reinforcement learning, values

Using graphical and pictorial representations to teach introductory astronomy students about the detection of extrasolar planets via gravitational microlensing

The detection and study of extrasolar planets is an exciting and thriving field in modern astrophysics, and an increasingly popular topic in introductory astronomy courses. One detection method relies on searching for stars whose light has been gravitationally microlensed by an extrasolar planet. In order to facilitate instructors' abilities to bring this interesting mix of general relativity and extrasolar planet detection into the introductory astronomy classroom, we have developed a new Lecture-Tutorial, "Detecting Exoplanets with Gravitational Microlensing." In this paper, we describe how this new Lecture-Tutorial's representations of astrophysical phenomena, which we selected and created based on theoretically motivated considerations of their pedagogical affordances, are used to help introductory astronomy students develop more expert-like reasoning abilities.Comment: 10 pages, 10 figures, accepted for publication in the American Journal of Physic

Analysing the visual dynamics of spatial morphology

Recently there has been a revival of interest in visibility analysis of architectural configurations. The new analyses rely heavily on computing power and statistical analysis, two factors which, according to the postpositivist school of geography, should immediately cause us to be wary. Thedanger, they would suggest, is in the application of a reductionist formal mathematical description in order to `explain' multilayered sociospatial phenomena. The author presents an attempt to rationalise how we can use visibility analysis to explore architecture in this multilayered context by considering the dynamics that lead to the visual experience. In particular, it is recommended that we assess the visualprocess of inhabitation, rather than assess the visibility in vacuo. In order to investigate the possibilities and limitations of the methodology, an urban environment is analysed by means of an agent-based model of visual actors within the configuration. The results obtained from the model are compared with actual pedestrian movement and other analytic measurements of the area: the agents correlate well both with human movement patterns and with configurational relationship as analysed by space-syntax methods. The application of both methods in combination improves on the correlation with observed movement of either, which in turn implies that an understanding of both the process of inhabitation and the principles of configuration may play a crucial role in determining the social usage of space