Automatic Understanding of Image and Video Advertisements

There is more to images than their objective physical content: for example, advertisements are created to persuade a viewer to take a certain action. We propose the novel problem of automatic advertisement understanding. To enable research on this problem, we create two datasets: an image dataset of 64,832 image ads, and a video dataset of 3,477 ads. Our data contains rich annotations encompassing the topic and sentiment of the ads, questions and answers describing what actions the viewer is prompted to take and the reasoning that the ad presents to persuade the viewer ("What should I do according to this ad, and why should I do it?"), and symbolic references ads make (e.g. a dove symbolizes peace). We also analyze the most common persuasive strategies ads use, and the capabilities that computer vision systems should have to understand these strategies. We present baseline classification results for several prediction tasks, including automatically answering questions about the messages of the ads.Comment: To appear in CVPR 2017; data available on http://cs.pitt.edu/~kovashka/ad

Machine Understanding of Human Behavior

A widely accepted prediction is that computing will move to the background, weaving itself into the fabric of our everyday living spaces and projecting the human user into the foreground. If this prediction is to come true, then next generation computing, which we will call human computing, should be about anticipatory user interfaces that should be human-centered, built for humans based on human models. They should transcend the traditional keyboard and mouse to include natural, human-like interactive functions including understanding and emulating certain human behaviors such as affective and social signaling. This article discusses a number of components of human behavior, how they might be integrated into computers, and how far we are from realizing the front end of human computing, that is, how far are we from enabling computers to understand human behavior

Let's start sewing

Master's Project (M.A.) University of Alaska Fairbanks, 2016This paper proposes Task Based Language Teaching (TBLT) as a teaching method for Ahtna language learners. TBLT focuses on engaging learners in meaningful activities or tasks which they accomplish through using the target language, learning Ahtna in the process. TBLT incorporates deeper understandings and meaning by teaching students the language in a cultural context. For this paper, the focus activity will be making a beaded necklace. Beading has been an important activity for me, from the time of learning about my culture and people from my Aunt Katie Wade. The website accompanying the project and be found at: http://www.ourlanguagecameback.com/

Early Social Interaction: A Case Comparison of Developmental Pragmatics and Psychoanalytic Theory

This book brings together various threads of the research work I have been involved with over a number of years. This research is based on a longitudinal video recorded study of one ofmydaughters as shewas learning howto talk. The impetus for engaging in this work arose from a sense that within developmental psychology and child language, when people are interested in understanding howchildren use language, they seem over-focused or concerned with questions of formal grammar and semantics. My interest is on understanding how a child learns to talk and through this process is then understood as being or becoming a member of a culture. When a young child is learning how to engage in everyday interaction she has to acquire those competencies that allow her to be simultaneously oriented to the conventions that inform talk-ininteraction and at the same time deal with the emotional or affective dimensions of her experience. It turns out that in developmental psychology these domains are traditionally studied separately or at least by researchers whose interests rarely overlap. In order to understand better early social relations (parent–child interaction), I want to pursue the idea that we will benefit by studying both early pragmatic development and emotional development. Not surprisingly, the theoretical positions underlying the study of these domains provide very different accounts of human development and this book illuminates why this might be the case. What follows will I hope serve as a case-study on the interdependence between the analysis of social interaction and subsequent interpretation

Modelling the influence of personality and culture on affect and enjoyment in multimedia

Affect is evoked through an intricate relationship between the characteristics of stimuli, individuals, and systems of perception. While affect is widely researched, few studies consider the combination of multimedia system characteristics and human factors together. As such, this paper explores the influence of personality (Five-Factor Model) and cultural traits (Hofstede Model) on the intensity of multimedia-evoked positive and negative affects (emotions). A set of 144 video sequences (from 12 short movie clips) were evaluated by 114 participants from a cross-cultural population, producing 1232 ratings. On this data, three multilevel regression models are compared: a baseline model that only considers system factors; an extended model that includes personality and culture; and an optimistic model in which each participant is modelled. An analysis shows that personal and cultural traits represent 5.6% of the variance in positive affect and 13.6% of the variance in negative affect. In addition, the affect-enjoyment correlation varied across the clips. This suggests that personality and culture play a key role in predicting the intensity of negative affect and whether or not it is enjoyed, but a more sophisticated set of predictors is needed to model positive affect with the same efficacy

Simulating emotional reactions in medical dramas

Presenting information on emotionally charged topics is a delicate task: if bare facts alone are conveyed, there is a risk of boring the audience, or coming across as cold and unfeeling; on the other hand, emotional presentation can be appropriate when carefully handled, but when overdone or mishandled risks being perceived as patronising or in poor taste. When Natural Language Generation (NLG) systems present emotionally charged information linguistically, by generating scripts for embodied agents, emotional/affective aspects cannot be ignored. It is important to ensure that viewers consider the presentation appropriate and sympathetic. We are investigating the role of affect in communicating medical information in the context of an NLG system that generates short medical dramas enacted by embodied agents. The dramas have both an informational and an educational purpose in that they help patients review their medical histories whilst receiving explanations of less familiar medical terms and demonstrations of their usage. The dramas are also personalised since they are generated from the patients' own medical records. We view generation of natural/appropriate emotional language as a way to engage and maintain the viewers' attention. For our medical setting, we hypothesize that viewers will consider dialogues more natural when they have an enthusiastic and sympathetic emotional tone. Our second hypothesis proposes that such dialogues are also better for engaging the viewers' attention. As well as describing our NLG system for generating natural emotional language in medical dialogue, we present a pilot study with which we investigate our two hypotheses. Our results were not quite as unequivocal as we had hoped. Firstly, our participants did notice whether a character sympathised with the patient and was enthusiastic. This did not, however, lead them to judge such a character as behaving more naturally or the dialogue as being more engaging. However, when pooling data from our two conditions, dialogues with versus dialogues without emotionally appropriate language use, we discovered, somewhat surprisingly, that participants did consider a dialogue more engaging if they believed that the characters showed sympathy towards the patient, were not cold and unfeeling, and were natural (true for the female agent only)

Making the Communication of CCS more "human"

CCS communication has proven a tough challenge, particularly for the difficulty in raising interest for the technology, which is still unknown to the majority of the population, and for the complexity of conveying information about its potential for reducing emissions. In this paper we present a research based effort for bringing CCS nearer to people, through visual material developed taking into account emotional needs related to the technology. The production of a short introductory film on CCS is illustrated and its testing with a sample of 700 high school students

Development and evaluation of a multimedia interactive CD: Public speaking interactive media

This paper reports on a study that endeavours to develop a Computer Assisted Learning (CAL) multimedia courseware namely, Public Speaking Interactive Media. This courseware was developed specifically for diploma students undergoing ENG4113 (Business English) and ENG 4153 (Public Speaking Skills) at Kolej Profesional MARA Indera Mahkota, Kuantan, Pahang. The objectives and goals of this study is to develop a CAL courseware which is in-line with the syllabus of the courses using multimedia elements together with the application of behaviorist, cognitive and constructivist learning theories as a basis in the design of the courseware. Moreover, the instructional design and implementation of this CAL multimedia courseware employ active and flexible learning strategies. Utilizing Hannafin and Peck’s Design Model, this courseware was developed using Macromedia Director and Macromedia Authorware to ensure that multimedia elements and simulations can be fully integrated. The findings of the study revealed that the courseware fulfilled its objectives in aiding students in comprehending the concept of public speaking skills better by using multimedia elements. In addition, the courseware is in-line with the syllabus and has incorporated the theories and strategies intended successfully

A Trip to the Moon: Personalized Animated Movies for Self-reflection

Self-tracking physiological and psychological data poses the challenge of presentation and interpretation. Insightful narratives for self-tracking data can motivate the user towards constructive self-reflection. One powerful form of narrative that engages audience across various culture and age groups is animated movies. We collected a week of self-reported mood and behavior data from each user and created in Unity a personalized animation based on their data. We evaluated the impact of their video in a randomized control trial with a non-personalized animated video as control. We found that personalized videos tend to be more emotionally engaging, encouraging greater and lengthier writing that indicated self-reflection about moods and behaviors, compared to non-personalized control videos