Choreographic and Somatic Approaches for the Development of Expressive Robotic Systems

As robotic systems are moved out of factory work cells into human-facing environments questions of choreography become central to their design, placement, and application. With a human viewer or counterpart present, a system will automatically be interpreted within context, style of movement, and form factor by human beings as animate elements of their environment. The interpretation by this human counterpart is critical to the success of the system's integration: knobs on the system need to make sense to a human counterpart; an artificial agent should have a way of notifying a human counterpart of a change in system state, possibly through motion profiles; and the motion of a human counterpart may have important contextual clues for task completion. Thus, professional choreographers, dance practitioners, and movement analysts are critical to research in robotics. They have design methods for movement that align with human audience perception, can identify simplified features of movement for human-robot interaction goals, and have detailed knowledge of the capacity of human movement. This article provides approaches employed by one research lab, specific impacts on technical and artistic projects within, and principles that may guide future such work. The background section reports on choreography, somatic perspectives, improvisation, the Laban/Bartenieff Movement System, and robotics. From this context methods including embodied exercises, writing prompts, and community building activities have been developed to facilitate interdisciplinary research. The results of this work is presented as an overview of a smattering of projects in areas like high-level motion planning, software development for rapid prototyping of movement, artistic output, and user studies that help understand how people interpret movement. Finally, guiding principles for other groups to adopt are posited.Comment: Under review at MDPI Arts Special Issue "The Machine as Artist (for the 21st Century)" http://www.mdpi.com/journal/arts/special_issues/Machine_Artis

The distracted robot: what happens when artificial agents behave like us

In everyday life, we are frequently exposed to different smart technologies. From our smartphones to avatars in computer games, and soon perhaps humanoid robots, we are surrounded by artificial agents created to interact with us. Already during the design phase of an artificial agent, engineers often endow it with functions aimed to promote the interaction and engagement with it, ranging from its \u201ccommunicative\u201d abilities to the movements it produces. Still, whether an artificial agent that can behave like a human could boost the spontaneity and naturalness of interaction is still an open question. Even during the interaction with conspecifics, humans rely partially on motion cues when they need to infer the mental states underpinning behavior. Similar processes may be activated during the interaction with embodied artificial agents, such as humanoid robots. At the same time, a humanoid robot that can faithfully reproduce human-like behavior may undermine the interaction, causing a shift in attribution: from being endearing to being uncanny. Furthermore, it is still not clear whether individual biases and prior knowledge related to artificial agents can override perceptual evidence of human-like traits. A relatively new area of research emerged in the context of investigating individuals\u2019 reactions towards robots, widely referred to as Human-Robot Interaction (HRI). HRI is a multidisciplinary community that comprises psychologists, neuroscientists, philosophers as well as roboticists, and engineers. However, HRI research has been often based on explicit measures (i.e. self-report questionnaires, a-posteriori interviews), while more implicit social cognitive processes that are elicited during the interaction with artificial agents took second place behind more qualitative and anecdotal results. The present work aims to demonstrate the usefulness of combining the systematic approach of cognitive neuroscience with HRI paradigms to further investigate social cognition processes evoked by artificial agents. Thus, this thesis aimed at exploring human sensitivity to anthropomorphic characteristics of a humanoid robot's (i.e. iCub robot) behavior, based on motion cues, under different conditions of prior knowledge. To meet this aim, we manipulated the human-likeness of the behaviors displayed by the robot and the explicitness of instructions provided to the participants, in both screen-based and real-time interaction scenarios. Furthermore, we explored some of the individual differences that affect general attitudes towards robots, and the attribution of human-likeness consequently

Advances in Human-Robot Interaction

Rapid advances in the field of robotics have made it possible to use robots not just in industrial automation but also in entertainment, rehabilitation, and home service. Since robots will likely affect many aspects of human existence, fundamental questions of human-robot interaction must be formulated and, if at all possible, resolved. Some of these questions are addressed in this collection of papers by leading HRI researchers

Aerospace medicine and biology: A continuing bibliography with indexes (supplement 324)

This bibliography lists 200 reports, articles and other documents introduced into the NASA Scientific and Technical Information System during May, 1989. Subject coverage includes: aerospace medicine and psychology, life support systems and controlled environments, safety equipment, exobiology and extraterrestrial life, and flight crew behavior and performance

Machine Performers: Agents in a Multiple Ontological State

In this thesis, the author explores and develops new attributes for machine performers and merges the trans-disciplinary fields of the performing arts and artificial intelligence. The main aim is to redefine the term “embodiment” for robots on the stage and to demonstrate that this term requires broadening in various fields of research. This redefining has required a multifaceted theoretical analysis of embodiment in the field of artificial intelligence (e.g. the uncanny valley), as well as the construction of new robots for the stage by the author. It is hoped that these practical experimental examples will generate more research by others in similar fields. Even though the historical lineage of robotics is engraved with theatrical strategies and dramaturgy, further application of constructive principles from the performing arts and evidence from psychology and neurology can shift the perception of robotic agents both on stage and in other cultural environments. In this light, the relation between representation, movement and behaviour of bodies has been further explored to establish links between constructed bodies (as in artificial intelligence) and perceived bodies (as performers on the theatrical stage). In the course of this research, several practical works have been designed and built, and subsequently presented to live audiences and research communities. Audience reactions have been analysed with surveys and discussions. Interviews have also been conducted with choreographers, curators and scientists about the value of machine performers. The main conclusions from this study are that fakery and mystification can be used as persuasive elements to enhance agency. Morphologies can also be applied that tightly couple brain and sensorimotor actions and lead to a stronger stage presence. In fact, if this lack of presence is left out of human replicants, it causes an “uncanny” lack of agency. Furthermore, the addition of stage presence leads to stronger identification from audiences, even for bodies dissimilar to their own. The author demonstrates that audience reactions are enhanced by building these effects into machine body structures: rather than identification through mimicry, this causes them to have more unambiguously biological associations. Alongside these traits, atmospheres such as those created by a cast of machine performers tend to cause even more intensely visceral responses. In this thesis, “embodiment” has emerged as a paradigm shift – as well as within this shift – and morphological computing has been explored as a method to deepen this visceral immersion. Therefore, this dissertation considers and builds machine performers as “true” performers for the stage, rather than mere objects with an aura. Their singular and customized embodiment can enable the development of non-anthropocentric performances that encompass the abstract and conceptual patterns in motion and generate – as from human performers – empathy, identification and experiential reactions in live audiences

Fantasy and functionality : Surveying personal italo disco related musical experience as a metaphoric-affective conceptualization process

In this thesis, I study personal musical experience in the light of so-called embodied cognition. According to this research paradigm, meaning arises from our interaction with the physical world in a process where the human body has a mediating role. The process has been shown to be metaphorical by nature, due to which I base my introspective findings mostly on the conceptual metaphor theory (CMT) of Lakoff & Johnson. Interaction with environment invokes affects, which has consequences in terms of person’s preference for metaphoric content. The goal of my work is to investigate reasons for my penchant for certain kind of music. Addressing this involves scrutinising my own perceptions and feelings when listening to italo disco, thus, music that I like, and supporting the findings with theories and earlier research. Besides CMT and also traditional music analysis, I utilise, for example, the complementary theories of the sonic object and musical gestures in pointing out experiential entities from textural and timbral elements of an italo disco song. The entities (sonic objects) are expected to arise from an affective sensation of motion, a process where conceptual metaphor plays an important role. Therefore, at the second stage of the analysis, I demonstrate metaphors on the grounds of the sonic objects in an attempt of establishing at least a part of my personal “metaphorical repertoire”. Arnie Cox’s mimetic hypothesis and framework for the study of musical affect is applied particularly during the second part of the analysis. The results of the inquiry confirm my hypothesis about the central role of conceptual metaphor in the emergence of (personal) musical experience. The set of metaphors I succeeded to point out suggest that I’m inclined to music that communicates powerful and energetic, thus clearly arousing affects–music that responds to the need of indulging in fantasy.Tutkin työssäni yksilöllistä musiikkikokemusta niin sanotun kehollisen kognition paradigman kautta. Tämän lähestymistavan mukaan merkitykset syntyvät vuorovaikutuksessa fyysisen maailman kanssa. Keholla on tässä prosessissa välittävä rooli. Prosessi on osoittautunut metaforiseksi, minkä vuoksi pohjaan introspektiiviset havaintoni pääasiassa Lakoffin & Johnsonin käsitteelliseen metaforateoriaan. Ympäristön kanssa vuorovaikutuksessa oleminen herättää affekteja, millä on vaikutusta siihen, minkälaisen metaforasisällön henkilö laittaa etusijalle. Työni tavoitteena on tutkia syitä musiikkimieltymyksiini. Metodina käytän omien aistimusteni ja tunteideni tarkkailua kuunnellessani italodiskoa–musiikkia, joka vetoaa minuun. Tuen introspektion tuloksia eri teorioilla sekä aiemmalla tutkimuksella. Käsitteellisen metaforateorian ja esimerkiksi perinteisen musiikkianalyysin lisäksi käytän muun muassa niin sanotun ääniobjektin ja musiikillisten eleiden teorioita pyrkiessäni osoittamaan kokemusperäisiä kokonaisuuksia italodiskokappaleen tekstuurisista ja äänenvärillisistä osatekijöistä. Kokonaisuuksien (ääniobjektien) voidaan olettaa syntyvän affektiivisen liikkeen aistimuksen pohjalta. Tässä prosessissa käsitteellinen metafora on tärkeässä roolissa. Siksi analyysin toisessa vaiheessa osoitan ääniobjekteihin pohjautuvia käsitteellisiä metaforia tarkoituksenani esitellä osa henkilökohtaisesta ”metaforarepertuaaristani”. Arnie Coxin mimeettistä hypoteesia ja musiikillisen affektin viitekehystä sovelletaan varsinkin analyysin toisessa vaiheessa. Tutkimuksen tulokset vahvistavat hypoteesini käsitteellisen metaforan keskeisestä roolista (henkilökohtaisen) musiikkikokemuksen synnyssä. Osoittamani metaforakokoelma viittaa viehtymykseeni voimakkaita ja energisiä, selvästi kiihottavia affekteja viestivää musiikkia kohtaan–musiikkia, joka mahdollistaa heittäytymisen fantasian maailmaan

Technology-supported training of arm-hand skills in stroke

Impaired arm-hand performance is a serious consequence of stroke that is associated with reduced self-efficacy and poor quality of life. Task-oriented arm training is a therapy approach that is known to improve skilled arm-hand performance, even in chronic stages after stroke. At the start of this project, little knowledge had been consolidated regarding taskoriented arm training characteristics, especially in the field of technology-supported rehabilitation. The feasibility and effects of technology-supported client-centred task-oriented training on skilled arm-hand performance had not been investigated but to a very limited degree. Reviewing literature on rehabilitation and motor learning in stroke led to the identification of therapy oriented criteria for rehabilitation technology aiming to influence skilled arm-hand performance (chapter 2). Most rehabilitation systems reported in literature to date are robotic systems that are aimed at providing an engaging exercise environment and feedback on motor performance. Both, feedback and engaging exercises are important for motivating patients to perform a high number of exercise repetitions and prolonged training, which are important factors for motor learning. The review also found that current rehabilitation technology is focussed mainly on providing treatment at a function level, thereby improving joint range of motion, muscle strength and parameters such as movement speed and smoothness of movement during analytical movements. However, related research has found no effects of robot-supported training at the activity level. The review concluded that a challenge exists for upper extremity rehabilitation technology in stroke patients to also provide more patienttailored task-oriented arm-hand training in natural environments to support the learning of skilled arm-hand performance. Besides mapping the strengths of different technological solutions, the use of outcome measures and training protocols needs to become more standardized across similar interventions, in order to help determine which training solutions are most suitable for specific patient categories. Chapter 4 contributes towards such a standardization of outcome measurement. A concept is introduced which may guide the clinician/researcher to choose outcome measures for evaluating specific and generalized training effects. As an initial operationalization of this concept, 28 test batteries that have been used in 16 task-oriented training interventions were rated as to whether measurement components were measured by the test. Future research is suggested that elaborates the concept with information on the relative weighing of components in each test, with more test batteries (which may lead to additional components) and by adding more test properties into the concept (e.g. psychometric properties of the tests, possible floor- or ceiling effects). Task-oriented training is one of the training approaches that has been shown to be beneficial for skilled arm-hand performance after stroke. Important mechanisms for motor learning that are identified are patient motivation for such training, and the learning of efficient goaloriented movement strategies and task-specific problem solving. In this thesis we operationalize task-oriented training in terms of 15 components (chapter 3). A systematic review that included 16 randomized controlled trials using task-oriented training in stroke patients, evaluated the effects of these training components on skilled arm-hand performance. The number of training components used in an intervention aimed at improving arm-hand performance after stroke was not associated with the post-treatment effect size. Distributed practice and feedback were associated with the largest post-intervention effect sizes. Random practice and use of clear functional training goals were associated with the largest follow-up effect sizes. It may be that training components that optimize the storage of learned motor performance in the long-term memory are associated with larger treatment effects. Unfortunately, feedback, random practice and distributed practice were reported in very few of the included randomized controlled trials (in only 6,3 and 1 out of the 17 studies respectively). Client-centred training, i.e. training on exercises that support goals that are selected by the patients themselves, improves patient motivation for training. Motivation in turn has proven to positively influence motor learning in stroke patients, as attention during training is heightened and storage of information in the long-term memory improves. Chapter 5 reports on an interview of 40 stroke patients, investigating into training preferences. A list of 46 skills, ranked according to descending training preference scores, was provided that can be used for implementation of exercises in rehabilitation technology, in order for technologysupported training to be client-centred. Chapter 6 introduces T-TOAT, a technology supported task-oriented arm training method that was developed together with colleagues at Adelante (Hoensbroek, NL). T-TOAT enables the implementation of exercises that support task-oriented training in rehabilitation technology. The training method is applicable for different technological systems, e.g. robot and sensor systems, or in combination with functional electrical stimulation, etc. To enable the use of TTOAT for training with the Haptic Master Robot (MOOG-FCS, NL), special software named Haptic TOAT was developed in Adelante together with colleagues at the Centre of Technology in Care of Zuyd University (chapter 6). The software enables the recording of the patient’s movement trajectories, given task constraints and patient possibilities, using the Haptic Master as a recording device. A purpose-made gimbal was attached to the endeffector, leaving the hand free for the use and manipulating objects. The recorded movement can be replayed in a passive mode or in an active mode (active, active-assisted or activeresisted). Haptic feedback is provided when the patient deviates from the recorded movement trajectory, as the patient receives the sensation of bouncing into a wall, as well as feeling a spring that pulls him/her back to the recorded path. The diameter of the tunnel around the recorded trajectory (distance to the wall), and the spring force can be adjusted for each patient. An ongoing clinical trial in which chronic stroke patients train with Haptic-TOAT examines whether Haptic Master provides additional value compared to supporting the same exercises by video-instruction only. Together with Philips Research Europe (Eindhoven,Aachen), the T-TOAT method has been implemented in a sensor based prototype, called Philips Stroke Rehabilitation Exerciser. This system included movement tracking sensors and an exercise board interacting with real life objects. A very strong feature of the system is that feedback is provided to patients (real-time and after exercise performance), based on a comparison of the patient’s exercise performance to individual targets set by the therapist. Chapter 7 reports on a clinical trial investigating arm-hand treatment outcome and patient motivation for technology-supported task-oriented training in chronic stroke patients. It was found that 8 weeks of T-TOAT training improved arm-hand performance in chronic stroke patients significantly on Fugl-Meyer, Action Research Arm Test, and Motor Activity Log. An improvement was found in health-related quality of life. Training effects lasted at least 6 months post-training. Participants reported feeling intrinsically motivated and competent to use the system. The results of this study showed that T-TOAT is feasible. Despite the small number of stroke patients tested (n=9), significant and clinically relevant improvements in skilled arm-hand performance were found. In conclusion, this thesis has made several contributions. It motivated the need for clientcentred task-oriented training, which it has operationalized in terms of 15 components. Four of these 15 components were identified as most beneficial for the patient. A prioritized inventory of arm-hand training preferences of stroke patients was compiled by means of an interview study of 40 subacute and chronic stroke patients. T-TOAT, a method for technology-supported, client-centred, task-oriented training, was conceived and implemented in two target technologies (Haptic Master and Philips Stroke Rehabilitation Exerciser). Its feasibility was demonstrated in a clinical trial showing substantial and durable benefits for the stroke patients. Finally, the thesis contributes towards the standardization of outcome measures which is necessary for charting progress and guiding future developments of technology-supported stroke rehabilitation. Methodological considerations were discussed and several suggestions for future research were presented. The variety of treatment approaches and the various ways of support and challenge that are offered by existing rehabilitation technologies hold a large potential for offering a variety of extra training opportunities to stroke patients that may improve their arm-hand performance. Such solutions will be of increasing importance, to alleviate therapists and reduce economic pressure on the health care system, as the stroke incidence is increasing rapidly over the coming decades