Eye-tracking Film Music

Film music scholars, composers, directors and audiences have always implicitly believed that music can help determine the focus of an audience’s visual attention, but researchers have not as yet been able to prove this empirically. Eye-tracking research—the process of measuring either the point of gaze or the motion of the eyes—has grown exponentially in recent years. This paper reports on a foundational, empirical eye-tracking study that examined the effects of contextual musical attributes on visual attention, emotion and user experience during exploration tasks in moving images. Our results show that music is able to direct how we see by switching attention to target foci more quickly as well as lengthening fixations, and that music can also encourage greater exploration of visual scenes outside targets. Our work contributes the first step in understanding how music shapes visual attention using eye-tracking techniques. We encourage wider adoption of this approach which has the potential to enhance understanding of the complex processes of audiovisual perception in action

Effects of Narrative Structure and Salient Decision Points in Role Playing Games

This research-in-progress paper reports an experimental study that investigates two research problems: first, how does narrative structure impact the experience of role-playing games (RPGs)? And second, what are the effects of salient decision points on players’ perceptions of narrative structure and game play? Can players perceive branching narrative without salient decision points? Previous research literature was reviewed, two hypotheses were developed based on prior research, and an experiment was designed to test these two hypotheses. The findings will likely shed light on the development of narrative and feedback structures in RPGs and other allied fields, such as digital media, virtual reality, and human-computer interaction

Are You Seeing What I'm Seeing? An Eye-Tracking Evaluation of Dynamic Scenes

Based on the data from the 2006/7 multimedia exhibition, RePossessed, during which over 400 members of the public watched scenes from Hitchcock's Vertigo, this paper describes the basis of an approach to the use of eye-tracking techniques, visualisations, and metrics to measure the influence of directorial techniques on film viewers' experience. Used as part of a repertoire of tools, the visualisation and quantitative evaluation of eye movement data can provide an intuitive and accessible approach to the evaluation of moving image based media and allow the conventions, assumptions and intuitive practices of film-making to be examined

Telling interactive stories: A practice-based investigation into new media interactive storytelling

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.Telling Interactive Stories is a practice-based thesis, which theoretically and practically probes the field of digital fictional interactive storytelling. The submission takes the form of the interactive cinema installation Crossed Lines together with a written element of the thesis which interrogates historical, contextual, theoretical, technical and critical aspects of the field of interactive narrative using new media. Crossed Lines is an original fictional interactive AV piece, amalgamating multiform plots, a multi-screen viewing environment, an interactive interface and an interactive story navigation form. The installation tells the stories of nine characters in a way that the viewer can constantly explore and switch between all nine forms, using a telephone keypad and handset as an interface, and can simultaneously observe all characters’ presence between the nine remote locations. Several research methodologies are utilised to analyse and evaluate the installation. Quantitative methodologies include the use of user tracking systems where the computational output of the installation provides measurements and timings of user choices and behaviours. Qualitative methodologies include theoretical and visual analysis, and in depth analysis of user responses using interviews, questionnaires, video recordings and cuttingedge eye-tracking technologies

Presentation adaptation for multimodal interface systems: Three essays on the effectiveness of user-centric content and modality adaptation

The use of devices is becoming increasingly ubiquitous and the contexts of their users more and more dynamic. This often leads to situations where one communication channel is rather impractical. Text-based communication is particularly inconvenient when the hands are already occupied with another task. Audio messages induce privacy risks and may disturb other people if used in public spaces. Multimodal interfaces thus offer users the flexibility to choose between multiple interaction modalities. While the choice of a suitable input modality lies in the hands of the users, they may also require output in a different modality depending on their situation. To adapt the output of a system to a particular context, rules are needed that specify how information should be presented given the users’ situation and state. Therefore, this thesis tests three adaptation rules that – based on observations from cognitive science – have the potential to improve the interaction with an application by adapting the presented content or its modality. Following modality alignment, the output (audio versus visual) of a smart home display is matched with the user’s input (spoken versus manual) to the system. Experimental evaluations reveal that preferences for an input modality are initially too unstable to infer a clear preference for either interaction modality. Thus, the data shows no clear relation between the users’ modality choice for the first interaction and their attitude towards output in different modalities. To apply multimodal redundancy, information is displayed in multiple modalities. An application of the rule in a video conference reveals that captions can significantly reduce confusion. However, the effect is limited to confusion resulting from language barriers, whereas contradictory auditory reports leave the participants in a state of confusion independent of whether captions are available or not. We therefore suggest to activate captions only when the facial expression of a user – captured by action units, expressions of positive or negative affect, and a reduced blink rate – implies that the captions effectively improve comprehension. Content filtering in movies puts the character into the spotlight that – according to the distribution of their gaze to elements in the previous scene – the users prefer. If preferences are predicted with machine learning classifiers, this has the potential to significantly improve the user’ involvement compared to scenes of elements that the user does not prefer. Focused attention is additionally higher compared to scenes in which multiple characters take a lead role