Categorizing identity from facial motion

Advances in marker-less motion capture technology now allow the accurate replication of facial motion and deformation in computer-generated imagery (CGI). A forced-choice discrimination paradigm using such CGI facial animations showed that human observers can categorize identity solely from facial motion cues. Animations were generated from motion captures acquired during natural speech, thus eliciting both rigid (head rotations and translations) and nonrigid (expressional changes) motion. To limit interferences from individual differences in facial form, all animations shared the same appearance. Observers were required to discriminate between different videos of facial motion and between the facial motions of different people. Performance was compared to the control condition of orientation-inverted facial motion. The results show that observers are able to make accurate discriminations of identity in the absence of all cues except facial motion. A clear inversion effect in both tasks provided consistency with previous studies, supporting the configural view of human face perception. The accuracy of this motion capture technology thus allowed stimuli to be generated that closely resembled real moving faces. Future studies may wish to implement such methodology when studying human face perception

Facial Expression Recognition of Instructor Using Deep Features and Extreme Learning Machine

Classroom communication involves teacher’s behavior and student’s responses. Extensive research has been done on the analysis of student’s facial expressions, but the impact of instructor’s facial expressions is yet an unexplored area of research. Facial expression recognition has the potential to predict the impact of teacher’s emotions in a classroom environment. Intelligent assessment of instructor behavior during lecture delivery not only might improve the learning environment but also could save time and resources utilized in manual assessment strategies. To address the issue of manual assessment, we propose an instructor’s facial expression recognition approach within a classroom using a feedforward learning model. First, the face is detected from the acquired lecture videos and key frames are selected, discarding all the redundant frames for effective high-level feature extraction. Then, deep features are extracted using multiple convolution neural networks along with parameter tuning which are then fed to a classifier. For fast learning and good generalization of the algorithm, a regularized extreme learning machine (RELM) classifier is employed which classifies five different expressions of the instructor within the classroom. Experiments are conducted on a newly created instructor’s facial expression dataset in classroom environments plus three benchmark facial datasets, i.e., Cohn–Kanade, the Japanese Female Facial Expression (JAFFE) dataset, and the Facial Expression Recognition 2013 (FER2013) dataset. Furthermore, the proposed method is compared with state-of-the-art techniques, traditional classifiers, and convolutional neural models. Experimentation results indicate significant performance gain on parameters such as accuracy, F1-score, and recall

Sound symbolism, speech expressivity and crossmodality

The direct links existing between sound and meaning which characterize sound symbolism can be thought of as mainly related to two kinds of phenomena: sound iconicity and sound metaphors. The first refers to the mirror relations established between sound and meaning effects (Nobile, 2011) and the latter as coined by Fonagy (1983) refers to the relationships based on analogies between meaning and speech sound production characteristics. Four relevant codes to the study of sound symbolism phenomena have been mentioned in the phonetic literature: the frequency code (Ohala, 1994), the respiratory code, the effort code (Gussenhoven, 2002) and the sirenic code (Gussenhoven, 2016). In the present work sound symbolism is taken to be the basis of speech expressivity because the meaning effects attributed to the spoken mode by the listeners are thought to be based on the acoustic features of sounds deriving from the various articulatory maneuvers yielding breath, voice, noise, resonance and silence. Based on the impression caused by the acoustic features, listeners attribute physiological, physical, psychological and social characteristics to speakers. In this way, speech can be considered both expressive and impressive, because it is used to convey meaning effects but it also impress listeners. Both segmental and prosodic elements are used to express meaning effects in speech. Among the prosodic elements vocal quality settings have received less attention regarding speech expressive uses. We argue that the investigation of the expressive uses of voice quality settings can be better approached if these settings are grouped according to their shared acoustic output properties and vocal tract configurations. Results of experiments relating symbolic uses of vocal qualities to semantic, acoustic and visual features by means of multidimensional analysis are reported and the expressive and impressive roles of vocal quality settings in spoken communication are discussed in relation to motivated links between sound forms and meaning effects. KEY WORDS: sound and meaning;  sound symbolism; speech expressivity; voice quality; acoustic analysis; perceptual analysis

Towards a framework for socially interactive robots

250 p.En las últimas décadas, la investigación en el campo de la robótica social ha crecido considerablemente. El desarrollo de diferentes tipos de robots y sus roles dentro de la sociedad se están expandiendo poco a poco. Los robots dotados de habilidades sociales pretenden ser utilizados para diferentes aplicaciones; por ejemplo, como profesores interactivos y asistentes educativos, para apoyar el manejo de la diabetes en niños, para ayudar a personas mayores con necesidades especiales, como actores interactivos en el teatro o incluso como asistentes en hoteles y centros comerciales.El equipo de investigación RSAIT ha estado trabajando en varias áreas de la robótica, en particular,en arquitecturas de control, exploración y navegación de robots, aprendizaje automático y visión por computador. El trabajo presentado en este trabajo de investigación tiene como objetivo añadir una nueva capa al desarrollo anterior, la capa de interacción humano-robot que se centra en las capacidades sociales que un robot debe mostrar al interactuar con personas, como expresar y percibir emociones, mostrar un alto nivel de diálogo, aprender modelos de otros agentes, establecer y mantener relaciones sociales, usar medios naturales de comunicación (mirada, gestos, etc.),mostrar personalidad y carácter distintivos y aprender competencias sociales.En esta tesis doctoral, tratamos de aportar nuestro grano de arena a las preguntas básicas que surgen cuando pensamos en robots sociales: (1) ¿Cómo nos comunicamos (u operamos) los humanos con los robots sociales?; y (2) ¿Cómo actúan los robots sociales con nosotros? En esa línea, el trabajo se ha desarrollado en dos fases: en la primera, nos hemos centrado en explorar desde un punto de vista práctico varias formas que los humanos utilizan para comunicarse con los robots de una maneranatural. En la segunda además, hemos investigado cómo los robots sociales deben actuar con el usuario.Con respecto a la primera fase, hemos desarrollado tres interfaces de usuario naturales que pretenden hacer que la interacción con los robots sociales sea más natural. Para probar tales interfaces se han desarrollado dos aplicaciones de diferente uso: robots guía y un sistema de controlde robot humanoides con fines de entretenimiento. Trabajar en esas aplicaciones nos ha permitido dotar a nuestros robots con algunas habilidades básicas, como la navegación, la comunicación entre robots y el reconocimiento de voz y las capacidades de comprensión.Por otro lado, en la segunda fase nos hemos centrado en la identificación y el desarrollo de los módulos básicos de comportamiento que este tipo de robots necesitan para ser socialmente creíbles y confiables mientras actúan como agentes sociales. Se ha desarrollado una arquitectura(framework) para robots socialmente interactivos que permite a los robots expresar diferentes tipos de emociones y mostrar un lenguaje corporal natural similar al humano según la tarea a realizar y lascondiciones ambientales.La validación de los diferentes estados de desarrollo de nuestros robots sociales se ha realizado mediante representaciones públicas. La exposición de nuestros robots al público en esas actuaciones se ha convertido en una herramienta esencial para medir cualitativamente la aceptación social de los prototipos que estamos desarrollando. De la misma manera que los robots necesitan un cuerpo físico para interactuar con el entorno y convertirse en inteligentes, los robots sociales necesitan participar socialmente en tareas reales para las que han sido desarrollados, para así poder mejorar su sociabilida

Facial motion perception in autism spectrum disorder and neurotypical controls

This thesis was submitted for the degree of Doctor of Philosophy and was awarded by Brunel University LondonFacial motion provides an abundance of information necessary for mediating social communication. Emotional expressions, head rotations and eye-gaze patterns allow us to extract categorical and qualitative information from others (Blake & Shiffrar, 2007). Autism Spectrum Disorder (ASD) is a neurodevelopmental condition characterised by a severe impairment in social cognition. One of the causes may be related to a fundamental deficit in perceiving human movement (Herrington et al., (2007). This hypothesis was investigated more closely within the current thesis. In neurotypical controls, the visual processing of facial motion was analysed via EEG alpha waves. Participants were tested on their ability to discriminate between successive animations (exhibiting rigid and nonrigid motion). The appearance of the stimuli remained constant over trials, meaning decisions were based solely on differential movement patterns. The parieto-occipital region was specifically selective to upright facial motion while the occipital cortex responded similarly to natural and manipulated faces. Over both regions, a distinct pattern of activity in response to upright faces was characterised by a transient decrease and subsequent increase in neural processing (Girges et al., 2014). These results were further supported by an fMRI study which showed sensitivity of the superior temporal sulcus (STS) to perceived facial movements relative to inanimate and animate stimuli. The ability to process information from dynamic faces was assessed in ASD. Participants were asked to recognise different sequences, unfamiliar identities and genders from facial motion captures. Stimuli were presented upright and inverted in order to assess configural processing. Relative to the controls, participants with ASD were significantly impaired on all three tasks and failed to show an inversion effect (O'Brien et al., 2014). Functional neuroimaging revealed atypical activities in the visual cortex, STS and fronto-parietal regions thought to contain mirror neurons in participants with ASD. These results point to a deficit in the visual processing of facial motion, which in turn may partly cause social communicative impairments in ASD

Proceedings of the International Conference Sensory Motor Concepts in Language & Cognition

This volume contains selected papers of the 2008 annual conference of the German Association for Social Science Research on Japan (Vereinigung für sozialwissenschaftliche Japanforschung e.V. – VSJF). The academic meeting has addressed the issue of demographic change in Japan in comparison to the social developments of ageing in Germany and other member states of the European Union. The conference was organized by the Institute for Modern Japanese Studies at Heinrich-Heine-University of Duesseldorf and took place at the Mutter Haus in Kaiserswerth (an ancient part of Duesseldorf). Speakers from Germany, England, Japan and the Netherlands presented their papers in four sessions on the topics “Demographic Trends and Social Analysis”, “Family and Welfare Policies”, “Ageing Society and the Organization of Households” and “Demographic Change and the Economy”. Central to all transnational and national studies on demographic change is the question of how societies can be reconstructed and be made adaptive to these changes in order to survive as solidarity communities. The authors of this volume attend to this question by discussing on recent trends of social and economic restructuring and giving insight into new research developments such as in the area of households and housing, family care work, medical insurance, robot technology or the employment sector