Aesthetic Highlight Detection in Movies Based on Synchronization of Spectators’ Reactions.

Detection of aesthetic highlights is a challenge for understanding the affective processes taking place during movie watching. In this paper we study spectators’ responses to movie aesthetic stimuli in a social context. Moreover, we look for uncovering the emotional component of aesthetic highlights in movies. Our assumption is that synchronized spectators’ physiological and behavioral reactions occur during these highlights because: (i) aesthetic choices of filmmakers are made to elicit specific emotional reactions (e.g. special effects, empathy and compassion toward a character, etc.) and (ii) watching a movie together causes spectators’ affective reactions to be synchronized through emotional contagion. We compare different approaches to estimation of synchronization among multiple spectators’ signals, such as pairwise, group and overall synchronization measures to detect aesthetic highlights in movies. The results show that the unsupervised architecture relying on synchronization measures is able to capture different properties of spectators’ synchronization and detect aesthetic highlights based on both spectators’ electrodermal and acceleration signals. We discover that pairwise synchronization measures perform the most accurately independently of the category of the highlights and movie genres. Moreover, we observe that electrodermal signals have more discriminative power than acceleration signals for highlight detection

Towards Analyzing and Predicting the Experience of Live Performances with Wearable Sensing

We present an approach to interpret the response of audiences to live performances by processing mobile sensor data. We apply our method on three different datasets obtained from three live performances, where each audience member wore a single tri-axial accelerometer and proximity sensor embedded inside a smart sensor pack. Using these sensor data, we developed a novel approach to predict audience members’ self-reported experience of the performances in terms of enjoyment, immersion, willingness to recommend the event to others, and change in mood. The proposed method uses an unsupervised method to identify informative intervals of the event, using the linkage of the audience members’ bodily movements, and uses data from these intervals only to estimate the audience members’ experience. We also analyze how the relative location of members of the audience can affect their experience and present an automatic way of recovering neighborhood information based on proximity sensors. We further show that the linkage of the audience members’ bodily movements is informative of memorable moments which were later reported by the audience

Physiological synchrony in brain and body as a measure of attentional engagement

Attentional engagement – the emotional, cognitive and behavioral connection with information to which the attention is focused – is important in all settings where humans process information. Measures of attentional engagement could be helpful to, for instance, support teachers in online classrooms, or individuals working together in teams. This thesis aims to use physiological synchrony, the similarity in neurophysiological responses across individuals, as an implicit measure of attentional engagement. The research is divided into two parts: the first investigates how different attentional modulations affect physiological synchrony in brains and bodies, the second explores the feasibility of using physiological synchrony as a tool to monitor attention in real-life settings.In Part I, the effect of different manipulations of attention on physiological synchrony in brain and body is explored. We find that physiological synchrony does not only reflect attentional engagement when measured in the electroencephalogram (EEG), but also when measured in electrodermal activity (EDA) or heart rate. Moreover, we find that physiological synchrony can reflect both sensory and top-down variations in attention, where top-down focus of attention is best reflected by synchrony in EEG, and where emotionally salient events attracting attention are best reflected by EDA and heart rate. Part II transitions into the practical applications of physiological synchrony in real-life contexts. Wearables are employed to measure physiological synchrony in EDA and heart rate, demonstrating comparable accuracy to high-end lab-grade equipment. The research also incorporates machine learning techniques, showing that physiological synchrony can be combined with novel unsupervised learning algorithms. Finally, measurements in classrooms reveal that physiological synchrony can be successfully monitored in real-life settings.While the findings are promising, the thesis acknowledges limitations in terms of sufficient data that are required for robust monitoring of attentional engagement and in terms of limited variance in attention explained by physiological synchrony. To advance the field, future work should focus on the applied, methodological and ethical questions that remain unanswered

Algorithms and procedures to analyze physiological signals in psychophysiological research

This dissertation presents analytical techniques which allow more information to be derived from psychophysiological data than possible with traditional methods. The techniques include an implemented algorithm for chest strain-gauge respiration signal analysis and a permutation testing method for evaluating changes over time in physiological signals. These methods are applied to three data sets, each examining physiological correlates of emotional experience. In the first study physiological correlates of moods induced using music were identified, although respiration entrainment confounds the issue of whether mood or the music caused the observed patterns. The second study examined physiological responses while subjects watched an emotion-inducing movie under three emotion-regulation conditions; changes relating both to the movie scenes and condition were identified. Finally, the third study evaluated short term changes in heart rate while viewing words in terms of the type of word viewed and later word recall

Motion and emotion : Semantic knowledge for hollywood film indexing

Ph.DDOCTOR OF PHILOSOPH

Cognitive Collaboration Found in Cardiac Physiology : Study in Classroom Environment

It is known that periods of intense social interaction result in shared patterns in collaborators' physiological signals. However, applied quantitative research on collaboration is hindered due to scarcity of objective metrics of teamwork effectiveness. Indeed, especially in the domain of productive, ecologically-valid activity such as programming, there is a lack of evidence for the most effective, affordable and reliable measures of collaboration quality. In this study we investigate synchrony in physiological signals between collaborating computer science students performing pair-programming exercises in a class room environment. We recorded electrocardiography over the course of a 60 minute programming session, using lightweight physiological sensors. We employ correlation of heart-rate variability features to study social psychophysiological compliance of the collaborating students. We found evident physiological compliance in collaborating dyads' heart-rate variability signals. Furthermore, dyads' self-reported workload was associated with the physiological compliance. Our results show viability of a novel approach to field measurement using lightweight devices in an uncontrolled environment, and suggest that self-reported collaboration quality can be assessed via physiological signals.Peer reviewe

Detecting Moments of Stress from Measurements of Wearable Physiological Sensors

There is a rich repertoire of methods for stress detection using various physiological signals and algorithms. However, there is still a gap in research efforts moving from laboratory studies to real-world settings. A small number of research has verified when a physiological response is a reaction to an extrinsic stimulus of the participant’s environment in real-world settings. Typically, physiological signals are correlated with the spatial characteristics of the physical environment, supported by video records or interviews. The present research aims to bridge the gap between laboratory settings and real-world field studies by introducing a new algorithm that leverages the capabilities of wearable physiological sensors to detect moments of stress (MOS). We propose a rule-based algorithm based on galvanic skin response and skin temperature, combing empirical findings with expert knowledge to ensure transferability between laboratory settings and real-world field studies. To verify our algorithm, we carried out a laboratory experiment to create a “gold standard” of physiological responses to stressors. We validated the algorithm in real-world field studies using a mixed-method approach by spatially correlating the participant’s perceived stress, geo-located questionnaires, and the corresponding real-world situation from the video. Results show that the algorithm detects MOS with 84% accuracy, showing high correlations between measured (by wearable sensors), reported (by questionnaires and eDiary entries), and recorded (by video) stress events. The urban stressors that were identified in the real-world studies originate from traffic congestion, dangerous driving situations, and crowded areas such as tourist attractions. The presented research can enhance stress detection in real life and may thus foster a better understanding of circumstances that bring about physiological stress in humans