Evaluating Engagement in Digital Narratives from Facial Data

Engagement researchers indicate that the engagement level of people in a narrative has an influence on people's subsequent story-related attitudes and beliefs, which helps psychologists understand people's social behaviours and personal experience. With the arrival of multimedia, the digital narrative combines multimedia features (e.g. varying images, music and voiceover) with traditional storytelling. Research on digital narratives has been widely used in helping students gain problem-solving and presentation skills as well as supporting child psychologists investigating children's social understanding such as family/peer relationships through completing their digital narratives. However, there is little study on the effect of multimedia features in digital narratives on the engagement level of people. This research focuses on measuring the levels of engagement of people in digital narratives and specifically on understanding the media effect of digital narratives on people's engagement levels. Measurement tools are developed and validated through analyses of facial data from different age groups (children and young adults) in watching stories with different media features of digital narratives. Data sources used in this research include a questionnaire with Smileyometer scale and the observation of each participant's facial behaviours

Machine Understanding of Human Behavior

A widely accepted prediction is that computing will move to the background, weaving itself into the fabric of our everyday living spaces and projecting the human user into the foreground. If this prediction is to come true, then next generation computing, which we will call human computing, should be about anticipatory user interfaces that should be human-centered, built for humans based on human models. They should transcend the traditional keyboard and mouse to include natural, human-like interactive functions including understanding and emulating certain human behaviors such as affective and social signaling. This article discusses a number of components of human behavior, how they might be integrated into computers, and how far we are from realizing the front end of human computing, that is, how far are we from enabling computers to understand human behavior

The development and testing of a child-inspired advertising disclosure to alert children to digital and embedded advertising

Via three studies, this article aims to develop and test an advertising disclosure which is understandable for children (ages six to 12 years old) and which can alert them to different types of advertising in multiple media formats. First, cocreation workshops with 24 children (ages eight to 11 years old) were held to determine a selection of disclosure designs based on insights from the target group. Second, two eye-tracking studies among 32 children (ages six to 12 years old) were conducted to test which of these disclosure designs attracted the most attention when the disclosures were integrated into a media context. These studies led to the selection of the final advertising disclosure: a black rectangular graphic with the word Reclame! (i.e., Dutch for "Advertising!") in yellow letters. Finally, a two-by-two, between-subjects experimental study (disclosure design: existing versus child-inspired advertising disclosure; advertising format: brand placement versus online banner advertising) with 157 children (ages 10 and 11 years old) was performed to test the effectiveness of the child-inspired disclosure by comparing it with existing ones. This study not only showed that children recognized, understood, and liked the child-inspired disclosure better than the existing ones, but they were also better able to recognize advertising after exposure to this child-inspired advertising disclosure

Spotting Agreement and Disagreement: A Survey of Nonverbal Audiovisual Cues and Tools

While detecting and interpreting temporal patterns of non–verbal behavioral cues in a given context is a natural and often unconscious process for humans, it remains a rather difficult task for computer systems. Nevertheless, it is an important one to achieve if the goal is to realise a naturalistic communication between humans and machines. Machines that are able to sense social attitudes like agreement and disagreement and respond to them in a meaningful way are likely to be welcomed by users due to the more natural, efficient and human–centered interaction they are bound to experience. This paper surveys the nonverbal cues that could be present during agreement and disagreement behavioural displays and lists a number of tools that could be useful in detecting them, as well as a few publicly available databases that could be used to train these tools for analysis of spontaneous, audiovisual instances of agreement and disagreement

Looking Beyond a Clever Narrative: Visual Context and Attention are Primary Drivers of Affect in Video Advertisements

Emotion evoked by an advertisement plays a key role in influencing brand recall and eventual consumer choices. Automatic ad affect recognition has several useful applications. However, the use of content-based feature representations does not give insights into how affect is modulated by aspects such as the ad scene setting, salient object attributes and their interactions. Neither do such approaches inform us on how humans prioritize visual information for ad understanding. Our work addresses these lacunae by decomposing video content into detected objects, coarse scene structure, object statistics and actively attended objects identified via eye-gaze. We measure the importance of each of these information channels by systematically incorporating related information into ad affect prediction models. Contrary to the popular notion that ad affect hinges on the narrative and the clever use of linguistic and social cues, we find that actively attended objects and the coarse scene structure better encode affective information as compared to individual scene objects or conspicuous background elements.Comment: Accepted for publication in the Proceedings of 20th ACM International Conference on Multimodal Interaction, Boulder, CO, US

Assessing user experience of context-aware interfaces in a retail store

Context-awareness is becoming an essential functionality of mobile applications. However, it remains challenging to capture the contextual experience in innovation research, since early-stage technologies have not reached maturity to be implemented in a real-life context. Moreover, users have difficulty in evaluating implicit interactions with context-aware interfaces since imagination of users is limited. Assuming that context impacts user experience, virtual reality (VR) provides an untapped potential for the domain of innovation research. The aim of this study (in progress) is to investigate the potential of user tests in virtual reality (here virtual retail store) for human-computer interaction to better match the needs of users and designers. Initially, the mock-up has been implemented in a retail store with its context-awareness being simulated using the Wizard of Oz methodology (N = 18). This approach is found to be time-consuming and not sufficient for evaluating radical context-aware innovations

Aerospace medicine and biology: A continuing bibliography with indexes (supplement 359)

This bibliography lists 164 reports, articles and other documents introduced into the NASA Scientific and Technical Information System during Jan. 1992. Subject coverage includes: aerospace medicine and physiology, life support systems and man/system technology, protective clothing, exobiology and extraterrestrial life, planetary biology, and flight crew behavior and performance