A Model Predictive Approach to Control the Motion of a Virtual Player in the Mirror Game

PublishedIn this paper, we focus on the design of a feedback controller that drives a virtual player to follow or lead a human player in the mirror game. The movement of the end-effector of the virtual player is modeled by means of a feedback controlled Haken-Kelso-Bunz (HKB) oscillator or a damped harmonic oscillator, which is coupled with the observed motion of the human player measured in real time. A model predictive control algorithm is developed for the virtual player to generate humanlike trajectories while maintaining individual motor signature and guaranteeing bounded tracking error. Experimental results based on a prototype setup show the effectiveness of our strategy and its advantages over other existing algorithms.European Project AlterEgo FP7 ICT 2.9 - Cognitive Sciences and Robotic

Modeling Joint Improvisation between Human and Virtual Players in the Mirror Game

Joint improvisation is observed to emerge spontaneously among humans performing joint action tasks, and has been associated with high levels of movement synchrony and enhanced sense of social bonding. Exploring the underlying cognitive and neural mechanisms behind the emergence of joint improvisation is an open research challenge. This paper investigates the emergence of jointly improvised movements between two participants in the mirror game, a paradigmatic joint task example. A theoretical model based on observations and analysis of experimental data is proposed to capture the main features of their interaction. A set of experiments is carried out to test and validate the model ability to reproduce the experimental observations. Then, the model is used to drive a computer avatar able to improvise joint motion with a human participant in real time. Finally, a convergence analysis of the proposed model is carried out to confirm its ability to reproduce the emergence of joint movement between the participants

Adaptive tracking control of a virtual player in the mirror game

The coordination of interpersonal rhythmic movements is of great significance due to its potential relevance to human motor rehabilitation. In this paper we consider the problem of designing a controller able to drive a virtual player (VP) capable of imitating and following a human player in the mirror game [1]. The classic nonlinear HakenKelso-Bunz (HKB) model is adopted to describe the social motor coordination between two players. An adaptive control algorithm is developed and implemented on the HKB model to drive the VP. It is proven that the position error between the VP driven by our control algorithm and the human player is upper bounded during the game. Finally, experiments are conducted on a prototype set-up in order to evaluate the performance of the proposed control algorithm and compare it with other existing algorithms.European Project AlterEgo FP7 ICT 2.9 - Cognitive Sciences and Robotic

Design and validation of a virtual player for studying interpersonal coordination in the mirror game

This is the author accepted manuscript. The final version is available from the publisher via the DOI in this record.The mirror game has been recently proposed as a simple, yet powerful paradigm for studying interpersonal interactions. It has been suggested that a virtual partner able to play the game with human subjects can be an effective tool to affect the underlying neural processes needed to establish the necessary connections between the players, and also to provide new clinical interventions for rehabilitation of patients suffering from social disorders. Inspired by the motor processes of the central nervous system (CNS) and the musculoskeletal system in the human body, in this paper we develop a novel interactive cognitive architecture based on nonlinear control theory to drive a virtual player (VP) to play the mirror game with a human player (HP) in different configurations. Specifically, we consider two cases: the former where the VP acts as leader and the latter where it acts as follower. The crucial problem is to design a feedback control architecture capable of imitating and following or leading a human player in a joint action task. Movement of the end-effector of the VP is modeled by means of a feedback controlled Haken-Kelso-Bunz (HKB) oscillator, which is coupled with the observed motion of the HP measured in real time. To this aim, two types of control algorithms (adaptive control and optimal control) are used and implemented on the HKB model so that the VP can generate a human-like motion while satisfying certain kinematic constraints. A proof of convergence of the control algorithms is presented in the paper together with an extensive numerical and experimental validation of their effectiveness. A comparison with other existing designs is also discussed, showing the flexibility and the advantages of our control-based approach.This work was funded by the European Project AlterEgo FP7 ICT 2.9 - Cognitive Sciences and Robotics, Grant Number 600610

A Person-Centric Design Framework for At-Home Motor Learning in Serious Games

abstract: In motor learning, real-time multi-modal feedback is a critical element in guided training. Serious games have been introduced as a platform for at-home motor training due to their highly interactive and multi-modal nature. This dissertation explores the design of a multimodal environment for at-home training in which an autonomous system observes and guides the user in the place of a live trainer, providing real-time assessment, feedback and difficulty adaptation as the subject masters a motor skill. After an in-depth review of the latest solutions in this field, this dissertation proposes a person-centric approach to the design of this environment, in contrast to the standard techniques implemented in related work, to address many of the limitations of these approaches. The unique advantages and restrictions of this approach are presented in the form of a case study in which a system entitled the "Autonomous Training Assistant" consisting of both hardware and software for guided at-home motor learning is designed and adapted for a specific individual and trainer. In this work, the design of an autonomous motor learning environment is approached from three areas: motor assessment, multimodal feedback, and serious game design. For motor assessment, a 3-dimensional assessment framework is proposed which comprises of 2 spatial (posture, progression) and 1 temporal (pacing) domains of real-time motor assessment. For multimodal feedback, a rod-shaped device called the "Intelligent Stick" is combined with an audio-visual interface to provide feedback to the subject in three domains (audio, visual, haptic). Feedback domains are mapped to modalities and feedback is provided whenever the user's performance deviates from the ideal performance level by an adaptive threshold. Approaches for multi-modal integration and feedback fading are discussed. Finally, a novel approach for stealth adaptation in serious game design is presented. This approach allows serious games to incorporate motor tasks in a more natural way, facilitating self-assessment by the subject. An evaluation of three different stealth adaptation approaches are presented and evaluated using the flow-state ratio metric. The dissertation concludes with directions for future work in the integration of stealth adaptation techniques across the field of exergames.Dissertation/ThesisDoctoral Dissertation Computer Science 201

Design of a Virtual Player for Joint Improvisation with Humans in the Mirror Game

Joint improvisation is often observed among humans performing joint action tasks. Exploring the underlying cognitive and neural mechanisms behind the emergence of joint improvisation is an open research challenge. This paper investigates jointly improvised movements between two participants in the mirror game, a paradigmatic joint task example. First, experiments involving movement coordination of different dyads of human players are performed in order to build a human benchmark. No designation of leader and follower is given beforehand. We find that joint improvisation is characterized by the lack of a leader and high levels of movement synchronization. Then, a theoretical model is proposed to capture some features of their interaction, and a set of experiments is carried out to test and validate the model ability to reproduce the experimental observations. Furthermore, the model is used to drive a computer avatar able to successfully improvise joint motion with a human participant in real time. Finally, a convergence analysis of the proposed model is carried out to confirm its ability to reproduce joint movements between the participants.This work was supported by European Project AlterEgo FP7 ICT 2.9 - Cognitive 321 Sciences and Robotics, Grant Number 600610 (MdB, http://www.euromov.eu/alterego/project)