Looking Beyond a Clever Narrative: Visual Context and Attention are Primary Drivers of Affect in Video Advertisements

Emotion evoked by an advertisement plays a key role in influencing brand recall and eventual consumer choices. Automatic ad affect recognition has several useful applications. However, the use of content-based feature representations does not give insights into how affect is modulated by aspects such as the ad scene setting, salient object attributes and their interactions. Neither do such approaches inform us on how humans prioritize visual information for ad understanding. Our work addresses these lacunae by decomposing video content into detected objects, coarse scene structure, object statistics and actively attended objects identified via eye-gaze. We measure the importance of each of these information channels by systematically incorporating related information into ad affect prediction models. Contrary to the popular notion that ad affect hinges on the narrative and the clever use of linguistic and social cues, we find that actively attended objects and the coarse scene structure better encode affective information as compared to individual scene objects or conspicuous background elements.Comment: Accepted for publication in the Proceedings of 20th ACM International Conference on Multimodal Interaction, Boulder, CO, US

Affect Recognition in Ads with Application to Computational Advertising

Advertisements (ads) often include strongly emotional content to leave a lasting impression on the viewer. This work (i) compiles an affective ad dataset capable of evoking coherent emotions across users, as determined from the affective opinions of five experts and 14 annotators; (ii) explores the efficacy of convolutional neural network (CNN) features for encoding emotions, and observes that CNN features outperform low-level audio-visual emotion descriptors upon extensive experimentation; and (iii) demonstrates how enhanced affect prediction facilitates computational advertising, and leads to better viewing experience while watching an online video stream embedded with ads based on a study involving 17 users. We model ad emotions based on subjective human opinions as well as objective multimodal features, and show how effectively modeling ad emotions can positively impact a real-life application.Comment: Accepted at the ACM International Conference on Multimedia (ACM MM) 201

Vodcast Impact on Students\u27 Attitudes and Behavioral Intentions

Purpose: This paper uses structural equation modeling to assess the effectiveness of Vodcasts (video podcasts) as part of a university’s communication strategy with prospective students. Design/methodology/approach: Three theoretical models were tested using a structural equation model. Findings: We find that perceived informativeness, credibility, and irritation of the advertising are directly related to the value of the Vodcast advertising. However of those three factors, only the informativeness is directly related to the intent to take further action toward enrollment. In addition, while prior work has suggested that perceived entertainment of advertising positively influences its perceived value, we find that for these university Vodcasts, perceived entertainment is not a statistically significant factor. Research limitations/implications: The results suggest that for Vodcasts used for these purposes, less attention should be given to entertainment value, and more attention should be focused on providing useful information in a manner that is credible and not irritating to students. Originality/value: Vodcasts have become part of the Internet multimedia experience and have been integrated into universities’ web-based promotion strategies. While prior work has examined general advertising on the web, few studies have considered the impact of the interactive medium of Vodcasts on attitudes and behavioral intentions

Zapping index: Using smile to measure advertisement zapping likelihood

In marketing and advertising research, 'zapping' is defined as the action when a viewer stops watching a commercial. Researchers analyze users' behavior in order to prevent zapping which helps advertisers to design effective commercials. Since emotions can be used to engage consumers, in this paper, we leverage automated facial expression analysis to understand consumers' zapping behavior. Firstly, we provide an accurate moment-to-moment smile detection algorithm. Secondly, we formulate a binary classification problem (zapping/non-zapping) based on real-world scenarios, and adopt smile response as the feature to predict zapping. Thirdly, to cope with the lack of a metric in advertising evaluation, we propose a new metric called Zapping Index (ZI). ZI is a moment-to-moment measurement of a user's zapping probability. It gauges not only the reaction of a user, but also the preference of a user to commercials. Finally, extensive experiments are performed to provide insights and we make recommendations that will be useful to both advertisers and advertisement publishers

Personality in Computational Advertising: A Benchmark

In the last decade, new ways of shopping online have increased the possibility of buying products and services more easily and faster than ever. In this new context, personality is a key determinant in the decision making of the consumer when shopping. A person’s buying choices are influenced by psychological factors like impulsiveness; indeed some consumers may be more susceptible to making impulse purchases than others. Since affective metadata are more closely related to the user’s experience than generic parameters, accurate predictions reveal important aspects of user’s attitudes, social life, including attitude of others and social identity. This work proposes a highly innovative research that uses a personality perspective to determine the unique associations among the consumer’s buying tendency and advert recommendations. In fact, the lack of a publicly available benchmark for computational advertising do not allow both the exploration of this intriguing research direction and the evaluation of recent algorithms. We present the ADS Dataset, a publicly available benchmark consisting of 300 real advertisements (i.e., Rich Media Ads, Image Ads, Text Ads) rated by 120 unacquainted individuals, enriched with Big-Five users’ personality factors and 1,200 personal users’ pictures

Doctor of Philosophy

dissertationThe theme of my dissertation is users' opinion learning. We propose three different studies to learn users' opinion using various approaches and to address several important research questions. Firstly, in order to discover the significant factors that induce the rating differences from user-generated reviews, we first extract possible specific influences from the review, known as aspects, and then we propose an unsupervised aspect-based sentiment learning system that assigns sentiment scores to potential aspects. Based on the sentiment scores, we adopt linear regression models to identify the aspects that lead to the rating differences. Food quality, service, dessert and drink quality, location, value, and general opinion toward the restaurants are recognized as the main influential factors that cause the Yelp rating differences among chain restaurants. Secondly, to understand the impact of time reminder designs such as counting down clock, progressing bar indicator, and remaining number of advertisements reminder embedded in specific long and short advertisement videos, we propose a 4 by 2 between-subject experimental study with follow-up survey questions to collect user's opinions toward different temporal designs in the video. Thirdly, our study analyzes the advertisement video designs from the content level. We design the advertisement video with high and low content relevance levels with the desired video. A 2 by 2 betweensubject experimental study with follow-up survey questions is proposed. Results point out that advertisement videos with high content relevance levels can lead to shorter video iv duration perception and less negative attitudes toward the video, but can also diminish the effectiveness of the advertisement with users recalling fewer products and brands promoted in both longer and shorter advertisement videos