Machine Understanding of Human Behavior

A widely accepted prediction is that computing will move to the background, weaving itself into the fabric of our everyday living spaces and projecting the human user into the foreground. If this prediction is to come true, then next generation computing, which we will call human computing, should be about anticipatory user interfaces that should be human-centered, built for humans based on human models. They should transcend the traditional keyboard and mouse to include natural, human-like interactive functions including understanding and emulating certain human behaviors such as affective and social signaling. This article discusses a number of components of human behavior, how they might be integrated into computers, and how far we are from realizing the front end of human computing, that is, how far are we from enabling computers to understand human behavior

Automatic Measurement of Affect in Dimensional and Continuous Spaces: Why, What, and How?

This paper aims to give a brief overview of the current state-of-the-art in automatic measurement of affect signals in dimensional and continuous spaces (a continuous scale from -1 to +1) by seeking answers to the following questions: i) why has the field shifted towards dimensional and continuous interpretations of affective displays recorded in real-world settings? ii) what are the affect dimensions used, and the affect signals measured? and iii) how has the current automatic measurement technology been developed, and how can we advance the field

Application of Generalized Partial Volume Estimation for Mutual Information based Registration of High Resolution SAR and Optical Imagery

Mutual information (MI) has proven its effectiveness for automated multimodal image registration for numerous remote sensing applications like image fusion. We analyze MI performance with respect to joint histogram bin size and the employed joint histogramming technique. The affect of generalized partial volume estimation (GPVE) utilizing B-spline kernels with different histogram bin sizes on MI performance has been thoroughly explored for registration of high resolution SAR (TerraSAR-X) and optical (IKONOS-2) satellite images. Our experiments highlight possibility of an inconsistent MI behavior with different joint histogram bin size which gets reduced with an increase in order of B-spline kernel employed in GPVE. In general, bin size reduction and/or increasing B-spline order have a smoothing affect on MI surfaces and even the lowest order B-spline with a suitable histogram bin size can achieve same pixel level accuracy as achieved by the higher order kernels more consistently

Rare earth based nanostructured materials: Synthesis, functionalization, properties and bioimaging and biosensing applications

Rare earth based nanostructures constitute a type of functional materials widely used and studied in the recent literature. The purpose of this review is to provide a general and comprehensive overview of the current state of the art, with special focus on the commonly employed synthesis methods and functionalization strategies of rare earth based nanoparticles and on their different bioimaging and biosensing applications. The luminescent (including downconversion, upconversion and permanent luminescence) and magnetic properties of rare earth based nanoparticles, as well as their ability to absorb X-rays, will also be explained and connected with their luminescent, magnetic resonance and X-ray computed tomography bioimaging applications, respectively. This review is not only restricted to nanoparticles, and recent advances reported for in other nanostructures containing rare earths, such as metal organic frameworks and lanthanide complexes conjugated with biological structures, will also be commented on.European Union 267226Ministerio de Economía y Competitividad MAT2014-54852-

Dyadic Speech-based Affect Recognition using DAMI-P2C Parent-child Multimodal Interaction Dataset

Automatic speech-based affect recognition of individuals in dyadic conversation is a challenging task, in part because of its heavy reliance on manual pre-processing. Traditional approaches frequently require hand-crafted speech features and segmentation of speaker turns. In this work, we design end-to-end deep learning methods to recognize each person's affective expression in an audio stream with two speakers, automatically discovering features and time regions relevant to the target speaker's affect. We integrate a local attention mechanism into the end-to-end architecture and compare the performance of three attention implementations -- one mean pooling and two weighted pooling methods. Our results show that the proposed weighted-pooling attention solutions are able to learn to focus on the regions containing target speaker's affective information and successfully extract the individual's valence and arousal intensity. Here we introduce and use a "dyadic affect in multimodal interaction - parent to child" (DAMI-P2C) dataset collected in a study of 34 families, where a parent and a child (3-7 years old) engage in reading storybooks together. In contrast to existing public datasets for affect recognition, each instance for both speakers in the DAMI-P2C dataset is annotated for the perceived affect by three labelers. To encourage more research on the challenging task of multi-speaker affect sensing, we make the annotated DAMI-P2C dataset publicly available, including acoustic features of the dyads' raw audios, affect annotations, and a diverse set of developmental, social, and demographic profiles of each dyad.Comment: Accepted by the 2020 International Conference on Multimodal Interaction (ICMI'20

Affect and believability in game characters:a review of the use of affective computing in games

Virtual agents are important in many digital environments. Designing a character that highly engages users in terms of interaction is an intricate task constrained by many requirements. One aspect that has gained more attention recently is the effective dimension of the agent. Several studies have addressed the possibility of developing an affect-aware system for a better user experience. Particularly in games, including emotional and social features in NPCs adds depth to the characters, enriches interaction possibilities, and combined with the basic level of competence, creates a more appealing game. Design requirements for emotionally intelligent NPCs differ from general autonomous agents with the main goal being a stronger player-agent relationship as opposed to problem solving and goal assessment. Nevertheless, deploying an affective module into NPCs adds to the complexity of the architecture and constraints. In addition, using such composite NPC in games seems beyond current technology, despite some brave attempts. However, a MARPO-type modular architecture would seem a useful starting point for adding emotions