First impressions: A survey on vision-based apparent personality trait analysis

© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes,creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.Personality analysis has been widely studied in psychology, neuropsychology, and signal processing fields, among others. From the past few years, it also became an attractive research area in visual computing. From the computational point of view, by far speech and text have been the most considered cues of information for analyzing personality. However, recently there has been an increasing interest from the computer vision community in analyzing personality from visual data. Recent computer vision approaches are able to accurately analyze human faces, body postures and behaviors, and use these information to infer apparent personality traits. Because of the overwhelming research interest in this topic, and of the potential impact that this sort of methods could have in society, we present in this paper an up-to-date review of existing vision-based approaches for apparent personality trait recognition. We describe seminal and cutting edge works on the subject, discussing and comparing their distinctive features and limitations. Future venues of research in the field are identified and discussed. Furthermore, aspects on the subjectivity in data labeling/evaluation, as well as current datasets and challenges organized to push the research on the field are reviewed.Peer ReviewedPostprint (author's final draft

Gesture meaning needs speech meaning to denote – A case of speech-gesture meaning interaction

We deal with a yet untreated issue in debates about linguistic interaction, namely a particular multi-modal dimension of meaning-dependence. We argue that the shape interpretation of speech-accompanying iconic gestures is dependent on its co-occurrent speech. Since there is no prototypical solution for modeling such a dependence, we offer an approach to compute a gesture’s meaning as a function of its speech context

Multi-modal meaning – An empirically-founded process algebra approach

Humans communicate with different modalities. We offer an account of multi-modal meaning coordination, taking speech-gesture meaning coordination as a prototypical case. We argue that temporal synchrony (plus prosody) does not determine how to coordinate speech meaning and gesture meaning. Challenging cases are asynchrony and broadcasting cases, which are illustrated with empirical data. We propose that a process algebra account satisfies the desiderata. It models gesture and speech as independent but concurrent processes that can communicate flexibly with each other and exchange the same information more than once. The account utilizes the psi-calculus, allowing for agents, input-output-channels, concurrent processes, and data transport of typed lambda-terms. A multi-modal meaning is produced integrating speech meaning and gesture meaning into one semantic package. Two cases of meaning coordination are handled in some detail: the asynchrony between gesture and speech, and the broadcasting of gesture meaning across several dialogue contributions. This account can be generalized to other cases of multi-modal meaning

Grappling with movement models: performing arts and slippery contexts

The ways we leave, recognise, and interpret marks of human movement are deeply entwined with layerings of collective memory. Although we retroactively order chronological sediments to map shareable stories, our remediations often emerge unpredictably from a multidimensional mnemonic fabric: contemporary ideas can resonate with ancient aspirations and initiatives, and foreign fields of investigation can inform ostensibly unrelated endeavours. Such links reinforce the debunking of grand narratives, and resonate with quests for the new kinds of thinking needed to address the mix of living, technological, and semiotic systems that makes up our wider ecology. As a highly evolving field, movement-and-computing is exceptionally open to, and needy of, this diversity. This paper argues for awareness of the analytical apparatus we sometimes too unwittingly bring to bear on our research objects, and for the value of transdisciplinary and tangential thinking to diversify our research questions. With a view to seeking ways to articulate new, shareable questions rather than propose answers, it looks at wider questions of problem-framing. It emphasises the importance of - quite literally - grounding movement, of recognising its environmental implications and qualities. Informed by work on expressive gesture and creative use of instruments in domains including puppetry and music, this paper also insists on the complexity and heterogeneity of the research strands that are indissociably bound up in our corporeal-technological movement practices

Agents for educational games and simulations

This book consists mainly of revised papers that were presented at the Agents for Educational Games and Simulation (AEGS) workshop held on May 2, 2011, as part of the Autonomous Agents and MultiAgent Systems (AAMAS) conference in Taipei, Taiwan. The 12 full papers presented were carefully reviewed and selected from various submissions. The papers are organized topical sections on middleware applications, dialogues and learning, adaption and convergence, and agent applications

Effectiveness of manual gesture treatment on residual /r/ articulation errors

The functional speech sound disorder, American English /r/ articulation errors, presents a unique and confounding clinical challenge as therapy resistant residual errors persist into adolescence and adulthood in many cases. Finding paucity of empirical research for /r/ treatment, evidence-based practice (EBP) exploration in motor-related disorders informed clinical practice and research directions. This study investigated the efficacy of manual mimicry (a kinesthetic, gestural, and visual cue) in treating intractable /r/ errors in a young adult using a single subject ABAB design. Perceptual accuracy judgments of three types of listeners (experts, graduate clinician, and naïve listeners) indicated a positive treatment effect of manual mimicry cueing on vocalic /r/ sound productions. Electropalatograpy (EPG) outcome measures showed limited ability to accurately reflect perceptual changes quantitatively. These findings from an exploratory study provide initial evidence that perceptual saliency of /r/ productions may be potentially remediated using a kinesthetic, gestural, and visual cue during treatment

Systems, interactions and macrotheory

A significant proportion of early HCI research was guided by one very clear vision: that the existing theory base in psychology and cognitive science could be developed to yield engineering tools for use in the interdisciplinary context of HCI design. While interface technologies and heuristic methods for behavioral evaluation have rapidly advanced in both capability and breadth of application, progress toward deeper theory has been modest, and some now believe it to be unnecessary. A case is presented for developing new forms of theory, based around generic “systems of interactors.” An overlapping, layered structure of macro- and microtheories could then serve an explanatory role, and could also bind together contributions from the different disciplines. Novel routes to formalizing and applying such theories provide a host of interesting and tractable problems for future basic research in HCI