Article thumbnail

Embodied Gesture Processing: Motor-Based Integration of Perception and Action in Social Artificial Agents

By Amir Sadeghipour and Stefan Kopp


A close coupling of perception and action processes is assumed to play an important role in basic capabilities of social interaction, such as guiding attention and observation of others’ behavior, coordinating the form and functions of behavior, or grounding the understanding of others’ behavior in one’s own experiences. In the attempt to endow artificial embodied agents with similar abilities, we present a probabilistic model for the integration of perception and generation of hand-arm gestures via a hierarchy of shared motor representations, allowing for combined bottom-up and top-down processing. Results from human-agent interactions are reported demonstrating the model’s performance in learning, observation, imitation, and generation of gestures

Topics: Article
Publisher: Springer-Verlag
OAI identifier:
Provided by: PubMed Central

To submit an update or takedown request for this paper, please submit an Update/Correction/Removal Request.

Suggested articles


  1. (2010). A model for production, perception, and acquisition of actions in face-to-face communication. Cogn Process.
  2. A probabilistic model of motor resonance for embodied gesture perception.
  3. (1989). A tutorial on hidden Markov models and selected applications in speech recognition. In:
  4. (2004). Abstraction in recognition to solve the correspondence problem for robot imitation. In: TAROS;
  5. (2001). Action observation activates premotor and parietal areas in a somatotopic manner: an fMRI study.
  6. (2006). Being with virtual others: neural correlates of social interaction. Neuropsychologia
  7. Cognition and emotion in the brain. Selected topics of the international symposium on limbic and association cortical systems.
  8. (2007). Communicative hand gestures and object-directed hand movements activated the mirror neuron system. Soc Cogn Affect Neurosci.
  9. (2003). Computational approaches to motor learning by imitation. Philos Trans R Soc Lond.
  10. (2006). Discriminative and adaptive imitation in uni-manual and bi-manual tasks. Robot Auton Syst.
  11. (2009). Embodied communication: speakers’ gestures affect listeners’ actions. Cognition
  12. (2006). Gesture recognition using a marionette model and dynamic Bayesian networks DBNs. In: Campilho
  13. (2011). Hierarchical MOSAIC for movement generation.
  14. (2005). Humanhumanoid interaction: is a humanoid robot perceived as a human?. Humanoid Robot.
  15. (2002). Imitation as a dual-route process featuring predictive and learning components: a biologically plausible computational model. In: Imitation in animals and artifacts.
  16. (2007). Incremental learning of gestures by imitation in a humanoid robot. In:
  17. (2008). Incremental learning, clustering and hierarchy formation of whole body motion patterns using adaptive Hidden Markov chains. Int J Robot Res.
  18. (1998). Internal models in the cerebellum. Trends Cogn Sci.
  19. (2011). Learning actions from observations.
  20. Learning and reproduction of gestures by imitation. Robot Autom Mag
  21. Learning movement sequences from demonstration.
  22. (2007). Learning of gestures by imitation in a humanoid robot. In: Dautenhahn
  23. (2006). Mirror neurons and imitation: a computationally guided review. Neural Netw
  24. (1998). Mirror neurons and the simulation theory of mind-reading. Trends Cogn Sci.
  25. (2001). MOSAIC model for sensorimotor learning and control. Neural Comput.
  26. (1995). Motor facilitation during action observation: a magnetic stimulation study.
  27. (2005). Motor primitives in vertebrates and invertebrates. Current Opin Neurobiol.
  28. (2001). Movement observation affects movement execution in a simple response task. Acta Psychologica
  29. (1998). Multiple paired forward and inverse models for motor control. Neural Netw.
  30. (2004). Neural primitives for motion control.
  31. (2008). Origins of human communication.
  32. (2007). Perceptual resonance: actioninduced modulation of perception.
  33. (1999). Perceptual symbol systems. Behav Brain Sci.
  34. (2003). predictive perception of actions: a biologically inspired robotics architecture for imitation and learning. Conn Sci.
  35. (1993). Prior feedback: a Bayesian approach to maximum likelihood estimation. Comput Statistic
  36. Robust learning of arm trajectories through human demonstration. In:
  37. (2006). Robust trajectory learning and approximation for robot programming by demonstration. Robot Auton Syst.
  38. (2002). Schema design and implementation of the grasp-related mirror neuron system. Biol Cybern.
  39. (2004). Self-organization of distributedly represented multiple behavior schemata in a mirror system: reviews of robot experiments using RNNPB. Neural Netw.
  40. (2002). Sensory-motor primitives as a basis for imitation: Linking perception to action and biology to robotics. In: Dautenhahn
  41. (2010). Singleneuron responses in humans during execution and observation of actions. Current Biol.
  42. (2009). Social motorics—towards an embodied basis of social human-robot interaction. Human CenterRobot Syst.
  43. (2010). Social resonance and embodied coordination in face-toface conversation with artificial interlocutors. Speech Commun Special Issue Speech Face-to-Face Commun.
  44. (2004). Synthesizing multimodal utterances for conversational agents. Comput Animat Virtual Worlds.
  45. (2005). The case for motor involvement in perceiving conspecifics. Psychol Bull.
  46. (2007). The motor hierarchy: from kinematics to goals and intentions.
  47. (2001). The perception-behavior expressway: Automatic effects of social perception on social behavior. Adv Exp Soc Psychol.
  48. Towards a real-time Bayesian imitation system for a humanoid robot.
  49. (2004). Using movement and intentions to understand simple events. Cogn Sci.
  50. (2003). Using nonconscious behavioral mimicry to create affiliation and rapport. Psychol Sci.