332 research outputs found
Analyse et synthèse de mouvements théâtraux expressifs
This thesis addresses the analysis and generation of expressive movements for virtual human character. Based on previous results from three different research areas (perception of emotions and biological motion, automatic recognition of affect and computer character animation), a low-dimensional motion representation is proposed. This representation consists of the spatio-temporal trajectories of end-effectors (i.e., head, hands and feet), and pelvis. We have argued that this representation is both suitable and sufficient for characterizing the underlying expressive content in human motion, and for controlling the generation of expressive whole-body movements. In order to prove these claims, this thesis proposes: (i) A new motion capture database inspired by physical theory, which contains three categories of motion (locomotion, theatrical and improvised movements), has been built for several actors; (ii) An automatic classification framework has been designed to qualitatively and quantitatively assess the amount of emotion contained in the data. It has been shown that the proposed low-dimensional representation preserves most of the motion cues salient to the expression of affect and emotions; (iii) A motion generation system has been implemented, both for reconstructing whole-body movements from the low-dimensional representation, and for producing novel end-effector expressive trajectories. A quantitative and qualitative evaluation of the generated whole body motions shows that these motions are as expressive as the movements recorded from human actors.Cette thèse porte sur l'analyse et la génération de mouvements expressifs pour des personnages humains virtuels. Sur la base de résultats de l’état de l’art issus de trois domaines de recherche différents - la perception des émotions et du mouvement biologique, la reconnaissance automatique des émotions et l'animation de personnages virtuels - une représentation en faible dimension des mouvements constituée des trajectoires spatio-temporelles des extrémités des chaînes articulées (tête, mains et pieds) et du pelvis a été proposée. Nous avons soutenu que cette représentation est à la fois appropriée et suffisante pour caractériser le contenu expressif du mouvement humain et pour contrôler la génération de mouvements corporels expressifs. Pour étayer cette affirmation, cette thèse propose:i) une nouvelle base de données de capture de mouvements inspirée par la théorie du théâtre physique. Cette base de données contient des exemples de différentes catégories de mouvements (c'est-à-dire des mouvements périodiques, des mouvements fonctionnels, des mouvements spontanés et des séquences de mouvements théâtraux), produits avec des états émotionnels distincts (joie, tristesse, détente, stress et neutre) et interprétés par plusieurs acteurs.ii) Une étude perceptuelle et une approche basée classification automatique conçus pour évaluer qualitativement et quantitativement l'information liée aux émotions véhiculées et encodées dans la représentation proposée. Nous avons observé que, bien que de légères différences dans la performance aient été trouvées par rapport à la situation où le corps entier a été utilisé, notre représentation conserve la plupart des marqueurs de mouvement liés à l'expression de laffect et des émotions.iii) Un système de synthèse de mouvement capable : a) de reconstruire des mouvements du corps entier à partir de la représentation à faible dimension proposée et b) de produire de nouvelles trajectoires extrémités expressives (incluant la trajectoire du bassin). Une évaluation quantitative et qualitative des mouvements du corps entier générés montre que ces mouvements sont aussi expressifs que les mouvements enregistrés à partir d'acteurs humains
Bayesian Action–Perception Computational Model: Interaction of Production and Recognition of Cursive Letters
In this paper, we study the collaboration of perception and action representations involved in cursive letter recognition and production. We propose a mathematical formulation for the whole perception–action loop, based on probabilistic modeling and Bayesian inference, which we call the Bayesian Action–Perception (BAP) model. Being a model of both perception and action processes, the purpose of this model is to study the interaction of these processes. More precisely, the model includes a feedback loop from motor production, which implements an internal simulation of movement. Motor knowledge can therefore be involved during perception tasks. In this paper, we formally define the BAP model and show how it solves the following six varied cognitive tasks using Bayesian inference: i) letter recognition (purely sensory), ii) writer recognition, iii) letter production (with different effectors), iv) copying of trajectories, v) copying of letters, and vi) letter recognition (with internal simulation of movements). We present computer simulations of each of these cognitive tasks, and discuss experimental predictions and theoretical developments
Human-Robot Handshaking: A Review
For some years now, the use of social, anthropomorphic robots in various
situations has been on the rise. These are robots developed to interact with
humans and are equipped with corresponding extremities. They already support
human users in various industries, such as retail, gastronomy, hotels,
education and healthcare. During such Human-Robot Interaction (HRI) scenarios,
physical touch plays a central role in the various applications of social
robots as interactive non-verbal behaviour is a key factor in making the
interaction more natural. Shaking hands is a simple, natural interaction used
commonly in many social contexts and is seen as a symbol of greeting, farewell
and congratulations. In this paper, we take a look at the existing state of
Human-Robot Handshaking research, categorise the works based on their focus
areas, draw out the major findings of these areas while analysing their
pitfalls. We mainly see that some form of synchronisation exists during the
different phases of the interaction. In addition to this, we also find that
additional factors like gaze, voice facial expressions etc. can affect the
perception of a robotic handshake and that internal factors like personality
and mood can affect the way in which handshaking behaviours are executed by
humans. Based on the findings and insights, we finally discuss possible ways
forward for research on such physically interactive behaviours.Comment: Pre-print version. Accepted for publication in the International
Journal of Social Robotic
Recommended from our members
Soft Morphological Computation
Soft Robotics is a relatively new area of research, where progress in material science has powered the next generation of robots, exhibiting biological-like properties such as soft/elastic tissues, compliance, resilience and more besides. One of the issues when employing soft robotics technologies is the soft nature of the interactions arising between the robot and its environment. These interactions are complex, and the their dynamics are non-linear and hard to capture with known models. In this thesis we argue that complex soft interactions
can actually be beneficial to the robot, and give rise to rich stimuli which can be used for the resolution of robot tasks. We further argue that the usefulness of these interactions depends on statistical regularities, or structure, that appear in the stimuli. To this end, robots should appropriately employ their morphology and their actions, to influence the system-environment interactions such that structure can arise in the stimuli. In this thesis we show that learning processes can be used to perform such a task. Following this rationale, this thesis proposes and supports the theory of Soft Morphological Computation (SoMComp), by which a soft robot should appropriately condition, or ‘affect’, the soft interactions to improve the quality of the physical stimuli arising from it. SoMComp is composed of four main principles, i.e.: Soft Proprioception, Soft Sensing, Soft Morphology and Soft Actuation. Each of these principles is explored in the context of haptic object recognition or object handling in soft robots. Finally, this thesis provides an overview of this research and its future directions.AHDB CP17
Recommended from our members
Visual Dynamics Models for Robotic Planning and Control
For a robot to interact with its environment, it must perceive the world and understand how the world evolves as a consequence of its actions. This thesis studies a few methods that a robot can use to respond to its observations, with a focus on instances that can leverage visual dynamic models. In general, these are models of how the visual observations of a robot evolves as a consequence of its actions. This could be in the form of predictive models that directly predict the future in the space of image pixels, in the space of visual features extracted from these images, or in the space of compact learned latent representations. The three instances that this thesis studies are in the context of visual servoing, visual planning, and representation learning for reinforcement learning. In the first case, we combine learned visual features with learning single-step predictive dynamics models and reinforcement learning to learn visual servoing mechanisms. In the second case, we use a deterministic multi-step video prediction model to achieve various manipulation tasks through visual planning. In addition, we show that conventional video prediction models are unequipped to model uncertainty and multiple futures, which could limit the planning capabilities of the robot. To address this, we propose a stochastic video prediction model that is trained with a combination of variational losses, adversarial losses, and perceptual losses, and show that this model can predict futures that are more realistic, diverse, and accurate. Unlike the first two cases, in which the dynamics model is used to make predictions for decision-making, the third case learns the model solely for representation learning. We learn a stochastic sequential latent variable model to learn a latent representation, and then use it as an intermediate representation for reinforcement learning. We show that this approach improves final performance and sample efficiency
Classification et Caractérisation de l'Expression Corporelle des Emotions dans des Actions Quotidiennes
The work conducted in this thesis can be summarized into four main steps.Firstly, we proposed a multi-level body movement notation system that allows the description ofexpressive body movement across various body actions. Secondly, we collected a new databaseof emotional body expression in daily actions. This database constitutes a large repository of bodilyexpression of emotions including the expression of 8 emotions in 7 actions, combining video andmotion capture recordings and resulting in more than 8000 sequences of expressive behaviors.Thirdly, we explored the classification of emotions based on our multi-level body movement notationsystem. Random Forest approach is used for this purpose. The advantage of using RandomForest approach in our work is double-fold : 1) reliability of the classification model and 2) possibilityto select a subset of relevant features based on their relevance measures. We also comparedthe automatic classification of emotions with human perception of emotions expressed in differentactions. Finally, we extracted the most relevant features that capture the expressive content of themotion based on the relevance measure of features returned by the Random Forest model. Weused this subset of features to explore the characterization of emotional body expression acrossdifferent actions. A Decision Tree model was used for this purpose.Ce travail de thèse peut être résumé en quatre étapes principales. Premièrement, nousavons proposé un système d’annotation multi-niveaux pour décrire le mouvement corporel expressif dansdifférentes actions. Deuxièmement, nous avons enregistré une base de données de l’expression corporelledes émotions dans des actions quotidiennes. Cette base de données constitue un large corpus de comportementsexpressifs considérant l’expression de 8 émotions dans 7 actions quotidiennes, combinant à la fois lesdonnées audio-visuelle et les données de capture de mouvement et donnant lieu à plus que 8000 séquencesde mouvement expressifs. Troisièmement, nous avons exploré la classification des émotions en se basantsur notre système d’annotation multi-niveaux. L’approche des forêts aléatoires est utilisée pour cette fin. L’utilisationdes forêts aléatoires dans notre travail a un double objectif : 1) la fiabilité du modèle de classification,et 2) la possibilité de sélectionner un sous-ensemble de paramètres pertinents en se basant sur la mesured’importance retournée par le modèle. Nous avons aussi comparé la classification automatique des émotionsavec la perception humaine des émotions exprimées dans différentes actions. Finalement, nous avonsextrait les paramètres les plus pertinents qui retiennent l’expressivité du mouvement en se basant sur la mesured’importance retournée par le modèle des forêts aléatoires. Nous avons utilisé ce sous-ensemble deparamètres pour explorer la caractérisation de l’expression corporelle des émotions dans différentes actionsquotidiennes. Un modèle d’arbre de décision a été utilisé pour cette fin
Perception of Human Movement Based on Modular Movement Primitives
People can identify and understand human movement from very
degraded visual information without effort.
A few dots representing the position of the joints
are enough to induce a vivid and stable percept of the underlying movement.
Due to this ability, the realistic animation of 3D characters requires
great skill. Studying the constituents of movement that looks natural would not
only help these artists, but also bring better understanding of the
underlying information processing in the brain.
Analogous to the hurdles in animation, the efforts of roboticists reflect the
complexity of motion production: controlling the many degrees of freedom
of a body requires time-consuming computations.
Modularity is one strategy to address this problem:
Complex movement can be decomposed into simple primitives.
A few primitives can conversely be used to compose a
large number of movements.
Many types of movement primitives (MPs) have been proposed
on different levels of information processing hierarchy in
the brain.
MPs have mostly been proposed for movement production.
Yet, modularity based on primitives might similarly enable robust movement
perception.
For my thesis, I have conducted perceptual experiments based on
the assumption of a shared representation of perception and action
based on MPs.
The three different types of MPs I have investigated are
temporal MPs (TMP), dynamical MPs (DMP), and coupled
Gaussian process dynamical models (cGPDM).
The MP-models have been trained on natural movements
to generate new movements. I then perceptually validated these
artificial movements in different psychophysical experiments.
In all experiments I used a two-alternative forced choice paradigm,
in which human observers were presented a movement based on
motion-capturing data, and one generated by an MP-model.
They were then asked to chose the movement which they perceived
as more natural.
In the first experiment I investigated walking movements, and
found that, in line with previous results, faithful representation
of movement dynamics is more important than good reconstruction
of pose.
In the second experiment I investigated the role of prediction
in perception using reaching movements.
Here, I found that perceived naturalness of the predictions is
similar to the perceived naturalness of movements itself obtained
in the first experiment.
I have found that MP models are able to produce movement that looks natural,
with the TMP achieving the highest perceptual scores as well
highest predictiveness of perceived naturalness among the three
model classes, suggesting their suitability for a shared representation
of perception and action
Motion and emotion estimation for robotic autism intervention.
Robots have recently emerged as a novel approach to treating autism spectrum disorder (ASD). A robot can be programmed to interact with children with ASD in order to reinforce positive social skills in a non-threatening environment. In prior work, robots were employed in interaction sessions with ASD children, but their sensory and learning abilities were limited, while a human therapist was heavily involved in “puppeteering” the robot. The objective of this work is to create the next-generation autism robot that includes several new interactive and decision-making capabilities that are not found in prior technology. Two of the main features that this robot would need to have is the ability to quantitatively estimate the patient’s motion performance and to correctly classify their emotions. This would allow for the potential diagnosis of autism and the ability to help autistic patients practice their skills. Therefore, in this thesis, we engineered components for a human-robot interaction system and confirmed them in experiments with the robots Baxter and Zeno, the sensors Empatica E4 and Kinect, and, finally, the open-source pose estimation software OpenPose. The Empatica E4 wristband is a wearable device that collects physiological measurements in real time from a test subject. Measurements were collected from ASD patients during human-robot interaction activities. Using this data and labels of attentiveness from a trained coder, a classifier was developed that provides a prediction of the patient’s level of engagement. The classifier outputs this prediction to a robot or supervising adult, allowing for decisions during intervention activities to keep the attention of the patient with autism. The CMU Perceptual Computing Lab’s OpenPose software package enables body, face, and hand tracking using an RGB camera (e.g., web camera) or an RGB-D camera (e.g., Microsoft Kinect). Integrating OpenPose with a robot allows the robot to collect information on user motion intent and perform motion imitation. In this work, we developed such a teleoperation interface with the Baxter robot. Finally, a novel algorithm, called Segment-based Online Dynamic Time Warping (SoDTW), and metric are proposed to help in the diagnosis of ASD. Social Robot Zeno, a childlike robot developed by Hanson Robotics, was used to test this algorithm and metric. Using the proposed algorithm, it is possible to classify a subject’s motion into different speeds or to use the resulting SoDTW score to evaluate the subject’s abilities
- …