The Irresistible Animacy of Lively Artefacts

This thesis explores the perception of ‘liveliness’, or ‘animacy’, in robotically driven artefacts. This perception is irresistible, pervasive, aesthetically potent and poorly understood. I argue that the Cartesian rationalist tendencies of robotic and artificial intelligence research cultures, and associated cognitivist theories of mind, fail to acknowledge the perceptual and instinctual emotional affects that lively artefacts elicit. The thesis examines how we see artefacts with particular qualities of motion to be alive, and asks what notions of cognition can explain these perceptions. ‘Irresistible Animacy’ is our human tendency to be drawn to the primitive and strangely thrilling nature of experiencing lively artefacts. I have two research methodologies; one is interdisciplinary scholarship and the other is my artistic practice of building lively artefacts. I have developed an approach that draws on first-order cybernetics’ central animating principle of feedback-control, and second-order cybernetics’ concerns with cognition. The foundations of this approach are based upon practices of machine making to embody and perform animate behaviour, both as scientific and artistic pursuits. These have inspired embodied, embedded, enactive, and extended notions of cognition. I have developed an understanding using a theoretical framework, drawing upon literature on visual perception, behavioural and social psychology, puppetry, animation, cybernetics, robotics, interaction and aesthetics. I take as a starting point, the understanding that the visual cortex of the vertebrate eye includes active feature-detection for animate agents in our environment, and actively constructs the causal and social structure of this environment. I suggest perceptual ambiguity is at the centre of all animated art forms. Ambiguity encourages natural curiosity and interactive participation. It also elicits complex visceral qualities of presence and the uncanny. In the making of my own Lively Artefacts, I demonstrate a series of different approaches including the use of abstraction, artificial life algorithms, and reactive techniques

Encouraging the expression of the unspeakable : influence and agency in a robotic creature

Thesis (S.M.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2007.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Includes bibliographical references (p. 165-177) and index.The boundary between subject and object is becoming ever-the-more blurred by the creation of new types of computational objects. Especially when these objects take the form of robotic creatures do we get to question the powerful impact of the object on the person. Couple this with the expression of internal, unspoken experience through the making of non-speech sounds and we have a situation that demands new thoughts and new methodologies. This thesis works through these questions via the design and study of syngvab, a robotic marionette that moves in response to human non-speech vocal sounds. I draw from the world of puppetry and performing objects in the creation of syngvab the object and its stage, showing how this old tradition is directly relevant for the development of non-anthropomorphic, non-zoomorphic robotic creatures. I show how this mongrel of an object requires different methodologies of study, pulling from actor-network theory to examine syngvab in a symmetric manner with the human participants. The results of a case study interaction with syngvab support the contention that non-speech sounds as drawn out by a robotic creature are a potent means of exploring and investigating the unspeakable.by Nicholas A. Knouf.S.M

The thing that should not be: predictive coding and the uncanny valley in perceiving human and humanoid robot actions

Using functional magnetic resonance imaging (fMRI) repetition suppression, we explored the selectivity of the human action perception system (APS), which consists of temporal, parietal and frontal areas, for the appearance and/or motion of the perceived agent. Participants watched body movements of a human (biological appearance and movement), a robot (mechanical appearance and movement) or an android (biological appearance, mechanical movement). With the exception of extrastriate body area, which showed more suppression for human like appearance, the APS was not selective for appearance or motion per se. Instead, distinctive responses were found to the mismatch between appearance and motion: whereas suppression effects for the human and robot were similar to each other, they were stronger for the android, notably in bilateral anterior intraparietal sulcus, a key node in the APS. These results could reflect increased prediction error as the brain negotiates an agent that appears human, but does not move biologically, and help explain the ‘uncanny valley’ phenomenon

Motion Preference Learning

© 2011 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works.Presented at the American Control Conference, San Francisco, CA, June 2011.In order to control systems to meet subjective criteria, one would like to construct objective functions that accurately represent human preferences. To do this, we develop robust estimators based on convex optimization that, given empirical, pairwise comparisons between motions, produce both (1) objective functions that are compatible with the expressed preferences, and (2) global optimizers (i.e., “best motions”) for these functions. The approach is demonstrated with an example in which human and synthetic motions are compared

Shadows, touch and digital puppeteering: a media archaeological approach

Aims The practical aim of this research project is to create a multi-touch digital puppetry system that simulates shadow theatre environments and translates gestural acts of touch into live and expressive control of virtual shadow figures. The research is focussed on the qualities of movement achievable through the haptics of single and multi-touch control of the digital puppets in the simulation. An associated aim is to create a collaborative environment where multiple performers can control dynamic animation and scenography, and create novel visualisations and narratives. The conceptual aim is to link traditional and new forms of puppetry seeking cultural significance in the ‘remediation’ of old forms that avail themselves of new haptic resources and collaborative interfaces. The thesis evaluates related prior art where traditional worlds of shadow performance meet new media, digital projection and 3D simulation, in order to investigate how changing technical contexts transform the potential of shadows as an expressive medium. Methodology The thesis uses cultural analysis of relevant documentary material to contextualise the practical work by relating the media archaeology of 2D puppetry—shadows, shadowgraphs and silhouettes—to landmark work in real-time computer graphics and performance animation. The survey considers the work of puppeteers, animators, computer graphics specialists and media artists. Through practice and an experimental approach to critical digital creativity, the study provides practical evidence of multiple iterations of controllable physics-based animation delivering expressive puppet motion through touch and multiuser interaction. Video sequences of puppet movement and written observational analysis document the intangible aspects of animation in performance. Through re-animation of archival shadow puppets, the study presents an emerging artistic media archaeological method. The major element of this method has been the restoration of a collection of Turkish Karagöz Shadow puppets from the Institut International de la Marionnette (Charleville, France) into a playable digital form. Results The thesis presents a developing creative and analytical framework for digital shadow puppetry. It proposes a media archaeological method for working creatively with puppet archives that unlock the kinetic and expressive potential of restored figures. The interaction design introduces novel approaches to puppetry control systems—using spring networks—with objects under physics-simulation that demonstrate emergent expressive qualities. The system facilitates a dance of agency¹ between puppeteer and digital instrument. The practical elements have produced several software iterations and a tool-kit for generating elegant, nuanced multi-touch shadow puppetry. The study presents accidental discoveries—serendipitous benefits of open-ended practical exploration. For instance: the extensible nature of the control system means novel input—other than touch—can provide exciting potential for accessible user interaction, e.g. with gaze duration and eye direction. The study also identifies limitations including the rate of software change and obsolescence, the scope of physics-based animation and failures of simulation. Originality/value The work has historical value in that it documents and begins a media archaeology of digital puppetry, an animated phenomenon of increasing academic and commercial interest. The work is of artistic value providing an interactive approach to making digital performance from archival material in the domain of shadow theatre. The work contributes to the electronic heritage of existing puppetry collections. The study establishes a survey of digital puppetry, setting a research agenda for future studies. Work may proceed to digitise, rig and create collaborative and web-mediated touch-based motion control systems for 2D and 3D puppets. The present study thus provides a solid platform to restore past performances and create new work from old, near forgotten-forms. ¹ Following Andrew Pickering, puppetry is ‘a temporally extended back-and-forth dance of human and non-human agency in which activity and passivity on both sides are reciprocally intertwined’ PICKERING, A. 2010. Material Culture and the Dance of Agency. In: BEAUDRY, M. C. & HICKS, D. (eds.) Oxford Handbook of Material Culture Studies. Oxford University Press.

Meeting Trees Halfway: Environmental Encounters in Theatre and Performance

abstract: How do trees (live and representational) participate in our theatrical and performed encounters with them? If trees are not inherently scenic, as their treatment in language and on stage might reinforce, how can they be retheorized as agents and participants in dramatic encounters? Using Diana Taylor’s theory of scenario to understand embodied encounters, I propose an alternative approach to understanding environmental beings (like trees) called “synercentrism,” which takes as its central tenet the active, if not 100 percent “willed,” participation of both human and non-human beings. I begin by mapping a continuum from objecthood to agenthood to trace the different ways that plants and trees are used, represented, and included in our encounters. The continuum provides a framework that more comprehensively unpacks human-plant relationships. My dissertation addresses the rich variety of representations and embodiments by focusing on three central chapter topics: the history of tree representation and inclusion in dramatic literature and performance; interactions with living trees in gardens, parks, and other dramatic arenas; and individual plays and plants that have a particularly strong grasp on cultural imaginaries. Each chapter is followed by one or more corresponding case studies (the first chapter is followed by case studies on plants in musical theatre; the second on performing plants and collaborative performance events; and the last on the dance drama Memory Rings and the Methuselah tree). I conclude with a discussion of how the framework of synercentrism can aid in the disruption of terministic screens and facilitate reciprocal relationships with trees and other environmental agents.Dissertation/ThesisDoctoral Dissertation Theatre 201