Films, Affective Computing and Aesthetic Experience: Identifying Emotional and Aesthetic Highlights from Multimodal Signals in a Social Setting.

Over the last years, affective computing has been strengthening its ties with the humanities, exploring and building understanding of people’s responses to specific artistic multimedia stimuli. “Aesthetic experience” is acknowledged to be the subjective part of some artistic exposure, namely, the inner affective state of a person exposed to some artistic object. In this work, we describe ongoing research activities for studying the aesthetic experience of people when exposed to movie artistic stimuli. To do so, this work is focused on the definition of emotional and aesthetic highlights in movies and studies the people responses to them using physiological and behavioral signals, in a social setting. In order to examine the suitability of multimodal signals for detecting highlights, we initially evaluate a supervised highlight detection system. Further, in order to provide an insight on the reactions of people, in a social setting, during emotional and aesthetic highlights, we study two unsupervised systems. Those systems are able to (a) measure the distance among the captured signals of multiple people using the dynamic time warping algorithm and (b) create a reaction profile for a group of people that would be indicative of whether that group reacts or not at a given time. The results indicate that the proposed systems are suitable for detecting highlights in movies and capturing some form of social interactions across different movie genres. Moreover, similar social interactions during exposure to emotional and some types of aesthetic highlights, such as those corresponding to technical or lightening choices of the director, can be observed. The utilization of electrodermal activity measurements yields in better performances than those achieved when using acceleration measurements, whereas fusion of the modalities does not appear to be beneficial for the majority of the cases

Aesthetic Highlight Detection in Movies Based on Synchronization of Spectators’ Reactions.

Detection of aesthetic highlights is a challenge for understanding the affective processes taking place during movie watching. In this paper we study spectators’ responses to movie aesthetic stimuli in a social context. Moreover, we look for uncovering the emotional component of aesthetic highlights in movies. Our assumption is that synchronized spectators’ physiological and behavioral reactions occur during these highlights because: (i) aesthetic choices of filmmakers are made to elicit specific emotional reactions (e.g. special effects, empathy and compassion toward a character, etc.) and (ii) watching a movie together causes spectators’ affective reactions to be synchronized through emotional contagion. We compare different approaches to estimation of synchronization among multiple spectators’ signals, such as pairwise, group and overall synchronization measures to detect aesthetic highlights in movies. The results show that the unsupervised architecture relying on synchronization measures is able to capture different properties of spectators’ synchronization and detect aesthetic highlights based on both spectators’ electrodermal and acceleration signals. We discover that pairwise synchronization measures perform the most accurately independently of the category of the highlights and movie genres. Moreover, we observe that electrodermal signals have more discriminative power than acceleration signals for highlight detection

Multimodal Affect and Aesthetic Experience

The term “aesthetic experience” corresponds to the inner state of a person exposed to form and content of artistic objects. Exploring certain aesthetic values of artistic objects, as well as interpreting the aesthetic experience of people when exposed to art can contribute towards understanding (a) art and (b) people’s affective reactions to artwork. Focusing on different types of artistic content, such as movies, music, urban art and other artwork, the goal of this workshop is to enhance the interdisciplinary collaboration between affective computing and aesthetics researchers

Recognizing Induced Emotions of Movie Audiences: Are Induced and Perceived Emotions the Same?

Predicting the emotional response of movie audi- ences to affective movie content is a challenging task in affective computing. Previous work has focused on using audiovisual movie content to predict movie induced emotions. However, the relationship between the audience’s perceptions of the affective movie content (perceived emotions) and the emotions evoked in the audience (induced emotions) remains unexplored. In this work, we address the relationship between perceived and in- duced emotions in movies, and identify features and modelling approaches effective for predicting movie induced emotions. First, we extend the LIRIS-ACCEDE database by annotating perceived emotions in a crowd-sourced manner, and find that perceived and induced emotions are not always consistent. Second, we show that dialogue events and aesthetic highlights are effective predictors of movie induced emotions. In addition to movie based features, we also study physiological and be- havioural measurements of audiences. Our experiments show that induced emotion recognition can benefit from including temporal context and from including multimodal information. Our study bridges the gap between affective content analysis and induced emotion prediction

Recognizing Induced Emotions of Movie Audiences From Multimodal Information

Recognizing emotional reactions of movie audiences to affective movie content is a challenging task in affective computing. Previous research on induced emotion recognition has mainly focused on using audio-visual movie content. Nevertheless, the relationship between the perceptions of the affective movie content (perceived emotions) and the emotions evoked in the audiences (induced emotions) is unexplored. In this work, we studied the relationship between perceived and induced emotions of movie audiences. Moreover, we investigated multimodal modelling approaches to predict movie induced emotions from movie content based features, as well as physiological and behavioral reactions of movie audiences. To carry out analysis of induced and perceived emotions, we first extended an existing database for movie affect analysis by annotating perceived emotions in a crowd-sourced manner. We find that perceived and induced emotions are not always consistent with each other. In addition, we show that perceived emotions, movie dialogues, and aesthetic highlights are discriminative for movie induced emotion recognition besides spectators’ physiological and behavioral reactions. Also, our experiments revealed that induced emotion recognition could benefit from including temporal information and performing multimodal fusion. Moreover, our work deeply investigated the gap between affective content analysis and induced emotion recognition by gaining insight into the relationships between aesthetic highlights, induced emotions, and perceived emotions

Empowering Responsible Online Gambling by Real-time Persuasive Information Systems

Online gambling, unlike other mediums of problem- atic and addictive behaviours, such as tobacco and alcohol, offers unprecedented opportunities for building information systems that are able to monitor and understand a user’s behaviour in real-time and adapt persuasive messages and interactions that would fit their personal profile and usage context. Online gambling industry usually provides Application Programming Interfaces (APIs) meant mainly to enable third-party applications to network with their gambling services and enhance a user’s gambling experience. In this industrial practice and experience paper, we advocate that such API’s can also be used to retrieve gamblers’ online data, such as browsing and betting history, promotions and available offers and use it to build more intel- ligent and proactive responsible gambling information systems. We report on our industrial experience in this field and make the argument that data available for persuasive marketing and usability should, under specific usage conditions, also be made available for responsible gambling information systems. This principle would provide equal opportunities for both directions. We discuss the psychological foundations of our proposed solution and the risks and challenges typically found when building such a software-assisted intervention, persuasion and emotion regulation technology. We also shed light on its potential implications from the perspectives of social corporate responsibility and data protection. We finally propose a conceptual architecture to demonstrate our vision and explain how it can be implemented. In the wider context, the paper is meant to provide insights on building behavioural awareness and regulation information systems in relation to problematic digital media usage

Change blindness: eradication of gestalt strategies

Arrays of eight, texture-defined rectangles were used as stimuli in a one-shot change blindness (CB) task where there was a 50% chance that one rectangle would change orientation between two successive presentations separated by an interval. CB was eliminated by cueing the target rectangle in the first stimulus, reduced by cueing in the interval and unaffected by cueing in the second presentation. This supports the idea that a representation was formed that persisted through the interval before being 'overwritten' by the second presentation (Landman et al, 2003 Vision Research 43149–164]. Another possibility is that participants used some kind of grouping or Gestalt strategy. To test this we changed the spatial position of the rectangles in the second presentation by shifting them along imaginary spokes (by ±1 degree) emanating from the central fixation point. There was no significant difference seen in performance between this and the standard task [F(1,4)=2.565, p=0.185]. This may suggest two things: (i) Gestalt grouping is not used as a strategy in these tasks, and (ii) it gives further weight to the argument that objects may be stored and retrieved from a pre-attentional store during this task

Enabling Responsible Online Gambling by Real-time Persuasive Technologies

Online gambling, unlike other offline addiction forms, provides unprecedented opportunities for monitoring users’ behaviour in real-time, along with the ability to adapt persuasive interactions and messages that would match the gamblers usage and personal context. Online gambling industry usually offers Application Programming Interfaces (APIs) that are mainly intended to allow third-party applications to interact with their services and enhance user’s experience. In this paper, we claim that such API’s can also be utilised to retrieve gamblers’ online data, such as browsing and betting history and other available offers, and use it to build more proactive and intelligent responsible gambling systems. We report on our experience in this field and make the argument that the available data for persuasive marketing and usability should, under certain usage conditions, also be made available for responsible online gambling services. We discuss the psychological foundations of our proposed approach and the risks and challenges typically resulted when building such a software-assisted intervention, persuasion and emotion regulation technology. We also explain the potential impact of corporate social responsibility and data protection prospects. Furthermore, we explore the required principles that should be followed by the gambling industry for enabling responsible online gambling. We finally propose a conceptual architecture to show our vision and explain how it can be implemented. In the broader context, the paper is intended to provide insights on building behavioural awareness and regulation information systems related to problematic digital media usage. Keywords: Persuasive technologies, responsible online gambling, gambling data availability, corporate social responsibility

Leveraging analytics to produce compelling and profitable film content

Producing compelling film content profitably is a top priority to the long-term prosperity of the film industry. Advances in digital technologies, increasing availabilities of granular big data, rapid diffusion of analytic techniques, and intensified competition from user generated content and original content produced by Subscription Video on Demand (SVOD) platforms have created unparalleled needs and opportunities for film producers to leverage analytics in content production. Built upon the theories of value creation and film production, this article proposes a conceptual framework of key analytic techniques that film producers may engage throughout the production process, such as script analytics, talent analytics, and audience analytics. The article further synthesizes the state-of-the-art research on and applications of these analytics, discuss the prospect of leveraging analytics in film production, and suggest fruitful avenues for future research with important managerial implications