Tracking the affective state of unseen persons.

Emotion recognition is an essential human ability critical for social functioning. It is widely assumed that identifying facial expression is the key to this, and models of emotion recognition have mainly focused on facial and bodily features in static, unnatural conditions. We developed a method called affective tracking to reveal and quantify the enormous contribution of visual context to affect (valence and arousal) perception. When characters' faces and bodies were masked in silent videos, viewers inferred the affect of the invisible characters successfully and in high agreement based solely on visual context. We further show that the context is not only sufficient but also necessary to accurately perceive human affect over time, as it provides a substantial and unique contribution beyond the information available from face and body. Our method (which we have made publicly available) reveals that emotion recognition is, at its heart, an issue of context as much as it is about faces

A Trip to the Moon: Personalized Animated Movies for Self-reflection

Self-tracking physiological and psychological data poses the challenge of presentation and interpretation. Insightful narratives for self-tracking data can motivate the user towards constructive self-reflection. One powerful form of narrative that engages audience across various culture and age groups is animated movies. We collected a week of self-reported mood and behavior data from each user and created in Unity a personalized animation based on their data. We evaluated the impact of their video in a randomized control trial with a non-personalized animated video as control. We found that personalized videos tend to be more emotionally engaging, encouraging greater and lengthier writing that indicated self-reflection about moods and behaviors, compared to non-personalized control videos

A window on reality: perceiving edited moving images

Edited moving images entertain, inform, and coerce us throughout our daily lives, yet until recently, the way people perceive movies has received little psychological attention. We review the history of empirical investigations into movie perception and the recent explosion of new research on the subject using methods such as behavioral experiments, functional magnetic resonance imagery (fMRI) eye tracking, and statistical corpus analysis. The Hollywood style of moviemaking, which permeates a wide range of visual media, has evolved formal conventions that are compatible with the natural dynamics of attention and humans’ assumptions about continuity of space, time, and action. Identifying how people overcome the sensory differences between movies and reality provides an insight into how the same cognitive processes are used to perceive continuity in the real world

Feeling and Healing Eco-social Catastrophe: The Horrific Slipstream of Danis Goulet\u27s Wakening

Cree/Métis filmmaker Danis Goulet’s science fiction short Wakening (2013) is set in Canada’s near future, yet the film reveals a slipstream of time where viewers are invited to contemplate the horrors of ecosocial crises—future, past, and present. I argue Wakening, as futuristic ecohorror, produces horrific feelings in the moment of its viewing that are inevitably entangled with the past, inviting its audiences to experience the monstrous contexts of Indigenous lives across time. To articulate this temporal dynamism, I overlay two key conceptual understandings: Walter Benjamin’s critiques of Western progress and historicism, and Indigenous notions of a Native slipstream. When brought together in Wakening, which is inspired by the First Nations movement Idle No More, these concepts not only help expose the horror of Indigenous ecosocial crises wrought by colonial and neocolonial occupations but also draw our attention to the timelessness of Indigenous resistance in the face of such ecohorror. Ultimately, there are two significant implications in understanding Wakening as ecohorror of dynamic temporality. First, such a reading continues the important work of revisioning the theoretical and critical boundaries of Western cinema. Goulet’s play with audiences’ familiar expectations of horror’s invitations to the weird challenge us all to recalibrate our sense of generic cinematic representation and its purpose. Relatedly, such readings highlight film’s politics of emotion: its ability to generate “affective alliances” that can potentially help us all re-imagine our temporal and spatial engagements with the world at large

The cinematic Muthos

The Greek word muthos names the role of the viewer or the reader in the process of recognizing a given narrative form or structure. It constitutes an active work of composition, integration and synthesis of heterogeneous elements and the establishment of a dynamic identity to the story being presented. It consists also of a sort of “negotiation” between the expected and the unexpected elements of a narrative, the foreseeable and the unforeseen. Through it, the reader or spectator is able to sustain and calibrate a level of expectation that explains for much of the narrative tension and the fruition provided by it. Not everything is predictable and not everything is unpredictable. The narrative structure provides the viewer or reader with a number of macro and micro ranges of possibilities and, therefore, with the chance to discriminate between the congruity and the incongruity of any diegetic sequence (i.e., the occurrence of events or reactions within those ranges and those which fall outside them). Film theorists have introduced and developed several surveying devices that help us to better grasp how this muthos takes place in the case of film viewing as well as to realize the different levels, both diegetic and perceptual, in which this activity takes place. These devices are, for instance, the relation between point-glance and point-object focal perspectives, the relation between music and emotion, erotetic narrative, the narrative enthymeme and the relation between reading-for-the-story and reading-within-the system. The objective of this paper is to propose a panoramic and comparative view of these explanatory devices and to present and assess their common rationale. All of them describe the way films are basically constructed through complex and juxtaposed sequences consisting on the instauration of a given range of possibilities and the cropping out or selection of one of those possibilities. Another common feature lies on the way these devices attribute the viewer an active role in the extraction, elaboration and development of cinematic meaning. Film viewing appeals constantly to the memory and activates different common cognitive abilities in the spectator, namely, our common methods of decision-making. I shall expand this view by proposing other ways through which movies exert a controlled appeal to the spectators’ cognitive capabilities and to their collaboration in filling in the gaps. I shall assume the hypothesis that the spectator’s more or less conscious awareness of her active role in film viewing could lead her – as Ernst Gombrich suggested in the case of painting -“to experience something of the ‘thrill of making’ ” and thus contribute to the explanation of the power of contemporary movies.info:eu-repo/semantics/publishedVersio

What would Jaws do? The tyranny of film and the relationship between gaze and higher-level narrative film comprehension

What is the relationship between film viewers’ eye movements and their film comprehension? Typical Hollywood movies induce strong attentional synchrony—most viewers look at the same things at the same time. Thus, we asked whether film viewers’ eye movements would differ based on their understanding—the mental model hypothesis—or whether any such differences would be overwhelmed by viewers’ attentional synchrony—the tyranny of film hypothesis. To investigate this question, we manipulated the presence/absence of prior film context and measured resulting differences in film comprehension and eye movements. Viewers watched a 12-second James Bond movie clip, ending just as a critical predictive inference should be drawn that Bond’s nemesis, “Jaws,” would fall from the sky onto a circus tent. The No-context condition saw only the 12-second clip, but the Context condition also saw the preceding 2.5 minutes of the movie before seeing the critical 12-second portion. Importantly, the Context condition viewers were more likely to draw the critical inference and were more likely to perceive coherence across the entire 6 shot sequence (as shown by event segmentation), indicating greater comprehension. Viewers’ eye movements showed strong attentional synchrony in both conditions as compared to a chance level baseline, but smaller differences between conditions. Specifically, the Context condition viewers showed slightly, but significantly, greater attentional synchrony and lower cognitive load (as shown by fixation probability) during the critical first circus tent shot. Thus, overall, the results were more consistent with the tyranny of film hypothesis than the mental model hypothesis. These results suggest the need for a theory that encompasses processes from the perception to the comprehension of film

Towards assisting the decision-making process for content creators in cinematic virtual reality through the analysis of movie cuts and their influence on viewers' behavior

Virtual Reality (VR) is gaining popularity in recent years due to the commercialization of personal devices. VR is a new and exciting medium to tell stories, however, the development of Cinematic Virtual Reality (CVR) content is still in an exploratory phase. One of the main reasons is that in this medium the user has now total or partial control of the camera, therefore viewers create their own personal experiences by deciding what to see in every moment, which can potentially hinder the delivery of a pre-established narrative. In the particular case of transitions from one shot to another (movie cuts), viewers may not be aligned with the main elements of the scene placed by the content creator to convey the story. This can result in viewers missing key elements of the narrative. In this work, we explore recent studies that analyze viewers’ behavior during cinematic cuts in VR videos, and we discuss guidelines and methods which can help filmmakers with the decision-making process when filming and editing their movies

Finding a Basic Interpretive Unit through the Human Visual Perception and Cognition-A Comparison between Filmmakers and Audiences

The analysis method and paradigm of film have become a controversial topic in the data-driven era. Film, is not only an attractive industry that can achieve filmmakers’ imagination but has become a perfect stimulus to understand human being’s mental activity. The core research in this study is to examine the impact of filmmaking experience and the role of narrative denoters from filmmakers’ construction to audiences’ interpretation. Based on previous studies and integrating cognitive approaches, the thesis re-explores the nature and essence of film and proposes an alternative term - narrative denoter - which can be used as the indication of message exchanging between filmmakers and audiences. Using a released film that has a complete story to do the experiment, this study investigates the relationship between major, film interpretations, event segmentation, and audience’s preference. The result showed that filmmaking experience does not impact the interpretation of film; however the identification of the narrative denoter played an important role in film perception and cognition; apart from these, the audience’s preference did not correlate with film interpretation. With respect to this result, the narrative denoter can be indicator to demonstrate the message transitions from filmmaker to audience. Suggestions were made for future cognitive film studies on using the narrative denoters as a new analysis unit

Meanings of music in film from a cognitive perspective

Cognitive psychology, with its focus on mind and its processes, is one of the approaches to study film music. Although music alone is said to be already meaningful, it gains and transfers specific meanings in the film context. This article aims to contribute to understanding of what film music means and how these meanings are processed in the cross-modal perception of a film. A review of the selected empirical research on film music with regard to meaning is followed by a short overview of the Annabel J. Cohen’s Congruence-Association Model (CAM) of media cognition. The model provides a framework for the experiments’ results and encourages future interdisciplinary studies in this area