Towards a socially adaptive digital playground

We are working towards a socially adaptive digital playground for children. To this end, we are looking into nonverbal synchrony and other social signals as a measure of social behaviour and into ways to alter game dynamics to trigger and inhibit certain social behaviours. Our first results indicate that we can indeed influence social behaviours in a digital playground by changing game dynamics. Furthermore, our first results show that we will be able to sense some of these social behaviours using only computer vision techniques. I propose an iterative method for working towards a socially adaptive digital playground

What does not happen: quantifying embodied engagement using NIMI and self-adaptors

Previous research into the quantification of embodied intellectual and emotional engagement using non-verbal movement parameters has not yielded consistent results across different studies. Our research introduces NIMI (Non-Instrumental Movement Inhibition) as an alternative parameter. We propose that the absence of certain types of possible movements can be a more holistic proxy for cognitive engagement with media (in seated persons) than searching for the presence of other movements. Rather than analyzing total movement as an indicator of engagement, our research team distinguishes between instrumental movements (i.e. physical movement serving a direct purpose in the given situation) and non-instrumental movements, and investigates them in the context of the narrative rhythm of the stimulus. We demonstrate that NIMI occurs by showing viewers’ movement levels entrained (i.e. synchronised) to the repeating narrative rhythm of a timed computer-presented quiz. Finally, we discuss the role of objective metrics of engagement in future context-aware analysis of human behaviour in audience research, interactive media and responsive system and interface design

Leading and following with a virtual trainer

This paper describes experiments with a virtual fitness trainer capable of mutually coordinated interaction. The virtual human co-exercises along with the user, leading as well as following in tempo, to motivate the user and to influence the speed with which the user performs the exercises. In a series of three experiments (20 participants in total) we attempted to influence the users' performance by manipulating the (timing of the) exercise behavior of the virtual trainer. The results show that it is possible to do this implicitly, using only micro adjustments to its bodily behavior. As such, the system is a rst step in the direction of mutually coordinated bodily interaction for virtual humans

Audio-visual foreground extraction for event characterization

This paper presents a new method able to integrate audio and visual information for scene analysis in a typical surveillance scenario, using only one camera and one monaural microphone. Visual information is analyzed by a standard visual background/foreground (BG/FG) modelling module, enhanced with a novelty detection stage, and coupled with an audio BG/FG modelling scheme. The audiovisual association is performed on-line, by exploiting the concept of synchrony. Experimental tests carrying out classification and clustering of events show all the potentialities of the proposed approach, also in comparison with the results obtained by using the single modalities

Measuring stress and cognitive load effects on the perceived quality of a multimodal dialogue system

In this paper we present the results of a pilot study investigating the impact of stress and cognitive load on the perceived interaction quality of a multimodal dialogue system for crisis management. Four test subjects interacted with the system in four differently configured trials aiming to induce low/high levels of stress and cognitive load. To measure the level of stress and cognitive load physiological sensors and subjective ratings were collected. After each trial the subjects filled in an evaluation questionnaire regarding the system interaction quality. In the end we conducted an in-depth interview with each subject. The trials were recorded with a webcam to facilitate the behaviour analysis. Results showed that both factors have an influence on the way subjects perceived the interaction quality, whereas the cognitive load seems to have a higher impact. Further quantitative experiments are needed in order to validate the results and quantify the weight of each factor. \u