A novel cognitive architecture for a human-like virtual player in the mirror game

The so called mirror game, which in its simplest formulation involves two people mirroring each other's hand's movement, provides a paradigm to study social interaction. However, a customized virtual player can replace either of the two human participants and hopefully help with the rehabilitation of patients suffering from social disorders by regulating its kinematics. In this paper we investigate the coordination movement between an avatar (virtual player) and a human player in the mentioned game. A novel cognitive architecture is proposed to drive the motion of the virtual player so that it generates a human-like trajectory in two different experimental models. In order to achieve this objective, the Haken-Kelso-Bunz (HKB) equation is adopted to describe the social motor coordination between the virtual and the human player. In addition, both an adaptive algorithm for the coupling parameters in the HKB equation and a feedback controller are developed in order to guarantee human features for the virtual player in its kinematics. Finally, extensive experiments are conducted to validate the approach described above

Design and validation of a virtual player for studying interpersonal coordination in the mirror game

This is the author accepted manuscript. The final version is available from the publisher via the DOI in this record.The mirror game has been recently proposed as a simple, yet powerful paradigm for studying interpersonal interactions. It has been suggested that a virtual partner able to play the game with human subjects can be an effective tool to affect the underlying neural processes needed to establish the necessary connections between the players, and also to provide new clinical interventions for rehabilitation of patients suffering from social disorders. Inspired by the motor processes of the central nervous system (CNS) and the musculoskeletal system in the human body, in this paper we develop a novel interactive cognitive architecture based on nonlinear control theory to drive a virtual player (VP) to play the mirror game with a human player (HP) in different configurations. Specifically, we consider two cases: the former where the VP acts as leader and the latter where it acts as follower. The crucial problem is to design a feedback control architecture capable of imitating and following or leading a human player in a joint action task. Movement of the end-effector of the VP is modeled by means of a feedback controlled Haken-Kelso-Bunz (HKB) oscillator, which is coupled with the observed motion of the HP measured in real time. To this aim, two types of control algorithms (adaptive control and optimal control) are used and implemented on the HKB model so that the VP can generate a human-like motion while satisfying certain kinematic constraints. A proof of convergence of the control algorithms is presented in the paper together with an extensive numerical and experimental validation of their effectiveness. A comparison with other existing designs is also discussed, showing the flexibility and the advantages of our control-based approach.This work was funded by the European Project AlterEgo FP7 ICT 2.9 - Cognitive Sciences and Robotics, Grant Number 600610

Dynamic similarity promotes interpersonal coordination in joint-action

Human movement has been studied for decades and dynamic laws of motion that are common to all humans have been derived. Yet, every individual moves differently from everyone else (faster/slower, harder/smoother etc). We propose here an index of such variability, namely an individual motor signature (IMS) able to capture the subtle differences in the way each of us moves. We show that the IMS of a person is time-invariant and that it significantly differs from those of other individuals. This allows us to quantify the dynamic similarity, a measure of rapport between dynamics of different individuals' movements, and demonstrate that it facilitates coordination during interaction. We use our measure to confirm a key prediction of the theory of similarity that coordination between two individuals performing a joint-action task is higher if their motions share similar dynamic features. Furthermore, we use a virtual avatar driven by an interactive cognitive architecture based on feedback control theory to explore the effects of different kinematic features of the avatar motion on the coordination with human players

Modeling Joint Improvisation between Human and Virtual Players in the Mirror Game

Joint improvisation is observed to emerge spontaneously among humans performing joint action tasks, and has been associated with high levels of movement synchrony and enhanced sense of social bonding. Exploring the underlying cognitive and neural mechanisms behind the emergence of joint improvisation is an open research challenge. This paper investigates the emergence of jointly improvised movements between two participants in the mirror game, a paradigmatic joint task example. A theoretical model based on observations and analysis of experimental data is proposed to capture the main features of their interaction. A set of experiments is carried out to test and validate the model ability to reproduce the experimental observations. Then, the model is used to drive a computer avatar able to improvise joint motion with a human participant in real time. Finally, a convergence analysis of the proposed model is carried out to confirm its ability to reproduce the emergence of joint movement between the participants

A Model Predictive Approach to Control the Motion of a Virtual Player in the Mirror Game

PublishedIn this paper, we focus on the design of a feedback controller that drives a virtual player to follow or lead a human player in the mirror game. The movement of the end-effector of the virtual player is modeled by means of a feedback controlled Haken-Kelso-Bunz (HKB) oscillator or a damped harmonic oscillator, which is coupled with the observed motion of the human player measured in real time. A model predictive control algorithm is developed for the virtual player to generate humanlike trajectories while maintaining individual motor signature and guaranteeing bounded tracking error. Experimental results based on a prototype setup show the effectiveness of our strategy and its advantages over other existing algorithms.European Project AlterEgo FP7 ICT 2.9 - Cognitive Sciences and Robotic

An aesthetics of touch: investigating the language of design relating to form

How well can designers communicate qualities of touch? This paper presents evidence that they have some capability to do so, much of which appears to have been learned, but at present make limited use of such language. Interviews with graduate designer-makers suggest that they are aware of and value the importance of touch and materiality in their work, but lack a vocabulary to fully relate to their detailed explanations of other aspects such as their intent or selection of materials. We believe that more attention should be paid to the verbal dialogue that happens in the design process, particularly as other researchers show that even making-based learning also has a strong verbal element to it. However, verbal language alone does not appear to be adequate for a comprehensive language of touch. Graduate designers-makers’ descriptive practices combined non-verbal manipulation within verbal accounts. We thus argue that haptic vocabularies do not simply describe material qualities, but rather are situated competences that physically demonstrate the presence of haptic qualities. Such competencies are more important than groups of verbal vocabularies in isolation. Design support for developing and extending haptic competences must take this wide range of considerations into account to comprehensively improve designers’ capabilities

Trends in virtual reality technologies for the learning patient

NextMed convened the Medicine Meets Virtual Reality 22 (MMVR 22) conference in 2016. Since 1992, the conference has brought together a diverse group of researchers to share creative solutions for the evolving challenge of integrating virtual reality tools into medical education. Virtual reality (VR) and its enabling technologies utilize hardware and software to simulate environments and encounters where users can interact and learn. The MMVR 22 symposium proceedings contain projects that support a variety of learners: medical students, practitioners, soldiers, and patients. This report will contemplate the trends in virtual reality technologies for patients navigating their medical and healthcare learning. The learning patient seeks more than intervention; they seek prevention. From virtual humans and environments to motion sensors and haptic devices, patients are surrounded by increasingly rich and transformative data-driven tools. Applied data enables VR applications to simulate experience, predict health outcomes, and motivate new behavior. The MMVR 22 presents investigations into the usability of wearable devices, the efficacy of avatar inclusion, and the viability of multi-player gaming. With increasing need for individualized and scalable programming, only committed open source efforts will align instructional designers, technology integrators, trainers, and clinicians. Curriculum and InstructionCurriculum and Instructio