How to capture the heart ? Reviewing 20 years of emotion measurement in advertising.

In the latest decades, emotions have become an important research topic in all behavioral sciences, and not the least in advertising. Yet, advertising literature on how to measure emotions is not straightforward. The major aim of this article is to give an update on the different methods used for measuring emotions in advertising and to discuss their validity and applicability. We further draw conclusions on the relation between emotions and traditional measures of advertising effectiveness. We finally formulate recommendations on the use of the different methods and make suggestions for future research.Research; Emotions; Science; Advertising; Effectiveness; Recommendations;

Tune in to your emotions: a robust personalized affective music player

The emotional power of music is exploited in a personalized affective music player (AMP) that selects music for mood enhancement. A biosignal approach is used to measure listeners’ personal emotional reactions to their own music as input for affective user models. Regression and kernel density estimation are applied to model the physiological changes the music elicits. Using these models, personalized music selections based on an affective goal state can be made. The AMP was validated in real-world trials over the course of several weeks. Results show that our models can cope with noisy situations and handle large inter-individual differences in the music domain. The AMP augments music listening where its techniques enable automated affect guidance. Our approach provides valuable insights for affective computing and user modeling, for which the AMP is a suitable carrier application

Plug-in to fear: game biosensors and negative physiological responses to music

The games industry is beginning to embark on an ambitious journey into the world of biometric gaming in search of more exciting and immersive gaming experiences. Whether or not biometric game technologies hold the key to unlock the “ultimate gaming experience” hinges not only on technological advancements alone but also on the game industry’s understanding of physiological responses to stimuli of different kinds, and its ability to interpret physiological data in terms of indicative meaning. With reference to horror genre games and music in particular, this article reviews some of the scientific literature relating to specific physiological responses induced by “fearful” or “unpleasant” musical stimuli, and considers some of the challenges facing the games industry in its quest for the ultimate “plugged-in” experience

Pain detection with bioimpedance methodology from 3-dimensional exploration of nociception in a postoperative observational trial

Although the measurement of dielectric properties of the skin is a long-known tool for assessing the changes caused by nociception, the frequency modulated response has not been considered yet. However, for a rigorous characterization of the biological tissue during noxious stimulation, the bioimpedance needs to be analyzed over time as well as over frequency. The 3-dimensional analysis of nociception, including bioimpedance, time, and frequency changes, is provided by ANSPEC-PRO device. The objective of this observational trial is the validation of the new pain monitor, named as ANSPEC-PRO. After ethics committee approval and informed consent, 26 patients were monitored during the postoperative recovery period: 13 patients with the in-house developed prototype ANSPEC-PRO and 13 with the commercial device MEDSTORM. At every 7 min, the pain intensity was measured using the index of Anspec-pro or Medstorm and the 0-10 numeric rating scale (NRS), pre-surgery for 14 min and post-anesthesia for 140 min. Non-significant differences were reported for specificity-sensitivity analysis between ANSPEC-PRO (AUC = 0.49) and MEDSTORM (AUC = 0.52) measured indexes. A statistically significant positive linear relationship was observed between Anspec-pro index and NRS (r(2) = 0.15, p < 0.01). Hence, we have obtained a validation of the prototype Anspec-pro which performs equally well as the commercial device under similar conditions

Learning deep physiological models of affect

Feature extraction and feature selection are crucial phases in the process of affective modeling. Both, however, incorporate substantial limitations that hinder the development of reliable and accurate models of affect. For the purpose of modeling affect manifested through physiology, this paper builds on recent advances in machine learning with deep learning (DL) approaches. The efficiency of DL algorithms that train artificial neural network models is tested and compared against standard feature extraction and selection approaches followed in the literature. Results on a game data corpus — containing players’ physiological signals (i.e. skin conductance and blood volume pulse) and subjective self-reports of affect — reveal that DL outperforms manual ad-hoc feature extraction as it yields significantly more accurate affective models. Moreover, it appears that DL meets and even outperforms affective models that are boosted by automatic feature selection, for several of the scenarios examined. As the DL method is generic and applicable to any affective modeling task, the key findings of the paper suggest that ad-hoc feature extraction and selection — to a lesser degree — could be bypassed.The authors would like to thank Tobias Mahlmann for his work on the development and administration of the cluster used to run the experiments. Special thanks for proofreading goes to Yana Knight. Thanks also go to the Theano development team, to all participants in our experiments, and to Ubisoft, NSERC and Canada Research Chairs for funding. This work is funded, in part, by the ILearnRW (project no: 318803) and the C2Learn (project no. 318480) FP7 ICT EU projects.peer-reviewe

An Intra-Subject Approach Based on the Application of HMM to Predict Concentration in Educational Contexts from Nonintrusive Physiological Signals in Real-World Situations

Previous research has proven the strong influence of emotions on student engagement and motivation. Therefore, emotion recognition is becoming very relevant in educational scenarios, but there is no standard method for predicting students’ affects. However, physiological signals have been widely used in educational contexts. Some physiological signals have shown a high accuracy in detecting emotions because they reflect spontaneous affect-related information, which is fresh and does not require additional control or interpretation. Most proposed works use measuring equipment for which applicability in real-world scenarios is limited because of its high cost and intrusiveness. To tackle this problem, in this work, we analyse the feasibility of developing low-cost and nonintrusive devices to obtain a high detection accuracy from easy-to-capture signals. By using both inter-subject and intra-subject models, we present an experimental study that aims to explore the potential application of Hidden Markov Models (HMM) to predict the concentration state from 4 commonly used physiological signals, namely heart rate, breath rate, skin conductance and skin temperature. We also study the effect of combining these four signals and analyse their potential use in an educational context in terms of intrusiveness, cost and accuracy. The results show that a high accuracy can be achieved with three of the signals when using HMM-based intra-subject models. However, inter-subject models, which are meant to obtain subject-independent approaches for affect detection, fail at the same task.This research was partly supported by Spanish Ministry of Science, Innovation and Universities through projects PGC2018-096463-B-I00 and PGC2018-102279-B-I00 (MCIU/AEI/FEDER, UE)

Personality Assessment Using Biosignals and Human Computer Interaction applied to Medical Decision Making

Clinical decision-making for patients with multiple acute or chronic diseases (i.e. multimorbidity) is complex. There is often no ’right’ or optimal treatment due to the potentially harmful effects of multiple interactions between drugs and diseases. This makes it necessary to establish trade-offs between the benefits and risks of different treatment strategies. This means also that there may be high levels of risk and uncertainty when making decisions. One factor that can influence how decisions are made under conditions of risk and uncertainty is the decision maker’s personality. The studies of this dissertation used biosignals and eye-tracking methods and developed pointer tracking techniques to monitor human computer interaction to assess, using machine learning techniques, the individual personality of decision makers. Data acquisition systems were designed and prepared to collect and synchronize: 1) physiological data - electrocardiogram, blood volume pulse and electrodermal activity; 2) human-computer interaction data - pointer movements, eye tracking and pupil diameter; 3) decision-making task data; and 4) personality questionnaire’ results. A set of processing tools was developed to ensure the correct extraction of psychophysiologyrelated features that could manifest personality. These features were combined by several machine learning algorithms to predict the Big-Five personality traits: Openness, Conscientiousness, Extraversion, Agreeableness and Conscientiousness. The five personality traits were well modelled by, at least, one of the sets of features extracted. With a sample of 88 students, features from the pointer movements in online surveys predicted four personality traits with a mean squared error (MSE)<0.46. The blood volume pulse responses in a decision-making task trained in a distinct sample of 79 students predicted four personality traits with a MSE<0.49. The application of the personality models based on the pointer movements in the personality questionnaire in a sample of 12 medical doctors achieved a MSE<0.40 for three personality traits. These were the best results achieved in each context of this thesis. The outcomes of this work demonstrate the huge potential of broader models that predict personality through human behaviour, with possible application in a wide variety of fields, such as human resources, medical research studies or machine learning approaches

A complex physiology-based empirical usability evaluation method in practice

This paper outlines the INTERFACE usability evaluation methodology developed by researchers of our department. It is based on the simultaneous assessment of Heart Period Variability (HPV), Skin Conductance (SC), and other data. One of the highlights of this methodology is its capability to identify quality attributes of software elements with a time-resolution of only a few seconds: in particular cases it can assess 2- or 3-second events. The Department of Ergonomics and Psychology at the Budapest University of Technology and Economics carried out applied research projects assessing very various software. After these, we can show different types of typical software problems identified by our method. The method of analysis allows us not only to decide what types of problems are significant to the users; however, on the other hand, the method allows us to decide, to what extent the found problems and their assessed severity concern all the users in general, or how these things depend on the type and characteris tics of the users