Emotions in context: examining pervasive affective sensing systems, applications, and analyses

Pervasive sensing has opened up new opportunities for measuring our feelings and understanding our behavior by monitoring our affective states while mobile. This review paper surveys pervasive affect sensing by examining and considering three major elements of affective pervasive systems, namely; “sensing”, “analysis”, and “application”. Sensing investigates the different sensing modalities that are used in existing real-time affective applications, Analysis explores different approaches to emotion recognition and visualization based on different types of collected data, and Application investigates different leading areas of affective applications. For each of the three aspects, the paper includes an extensive survey of the literature and finally outlines some of challenges and future research opportunities of affective sensing in the context of pervasive computing

Wearable Computing for Health and Fitness: Exploring the Relationship between Data and Human Behaviour

Health and fitness wearable technology has recently advanced, making it easier for an individual to monitor their behaviours. Previously self generated data interacts with the user to motivate positive behaviour change, but issues arise when relating this to long term mention of wearable devices. Previous studies within this area are discussed. We also consider a new approach where data is used to support instead of motivate, through monitoring and logging to encourage reflection. Based on issues highlighted, we then make recommendations on the direction in which future work could be most beneficial

The Effects of Displayed Violence and Game Speed in First-Person Shooters on Physiological Arousal and Aggressive Behavior

Many studies have been conducted to examine the effects of displayed violence in digital games on outcomes like aggressive behavior and physiological arousal. However, they often lack a proper manipulation of the relevant factors and control of confounding variables. In this study, the displayed violence and game speed of a recent first-person shooter game were varied systematically using the technique of modding, so that effects could be explained properly by the respective manipulations. Aggressive behavior was measured with the standardized version of the Competitive Reaction Time Task or CRTT (Ferguson et al., 2008}. Physiological arousal was operationalized with four measurements: galvanic skin response (GSR), heart rate (HR), body movement, force on mouse and keyboard. A total of N = 87 participants played in one of four game conditions (low- vs. high-violence, normal- vs. high speed) while physiological measurements were taken with finger clips, force sensors on input devices (mouse and keyboard), and a Nintendo Wii balance board on the chair they sat on. After play, their aggressive behavior was measured with the CRTT. The results of the study do not support the hypothesis that playing digital games increases aggressive behavior. There were no significant differences in GSR and HR, but with a higher game speed, participants showed less overall body movement, most likely to meet the game’s higher demands on cognitive and motor capacities. Also, higher game speed and displayed violence caused an increase in applied force on mouse and keyboard. Previous experience with digital games did not moderate any of these findings. Moreover, it provides further evidence that the CRTT should only be used in a standardized way as a measurement for aggression, if at all. Using all 7 different published (though not validated) ways to calculate levels of aggression from the raw data, “evidence” was found that playing a violent digital game increases, decreases, or does not change aggression at all. Thus, the present study does extend previous research. Firstly, it shows the methodological advantages of modding in digital game research to accomplish the principles of psychological (laboratory) experiments by manipulating relevant variables and controlling all others. It also demonstrates the test-theoretical problems of the highly diverse use of the CRTT. It provides evidence that for a meaningful interpretation of effects of displayed violence in digital games, there are other game characteristics that should be controlled for since they might have an effect on relevant outcome variables. Further research needs to identify more of those game features, and it should also improve the understanding of the different measures for physiological arousal and their interrelatedness

Logging Stress and Anxiety Using a Gamified Mobile-based EMA Application, and Emotion Recognition Using a Personalized Machine Learning Approach

According to American Psychological Association (APA) more than 9 in 10 (94 percent) adults believe that stress can contribute to the development of major health problems, such as heart disease, depression, and obesity. Due to the subjective nature of stress, and anxiety, it has been demanding to measure these psychological issues accurately by only relying on objective means. In recent years, researchers have increasingly utilized computer vision techniques and machine learning algorithms to develop scalable and accessible solutions for remote mental health monitoring via web and mobile applications. To further enhance accuracy in the field of digital health and precision diagnostics, there is a need for personalized machine-learning approaches that focus on recognizing mental states based on individual characteristics, rather than relying solely on general-purpose solutions. This thesis focuses on conducting experiments aimed at recognizing and assessing levels of stress and anxiety in participants. In the initial phase of the study, a mobile application with broad applicability (compatible with both Android and iPhone platforms) is introduced (we called it STAND). This application serves the purpose of Ecological Momentary Assessment (EMA). Participants receive daily notifications through this smartphone-based app, which redirects them to a screen consisting of three components. These components include a question that prompts participants to indicate their current levels of stress and anxiety, a rating scale ranging from 1 to 10 for quantifying their response, and the ability to capture a selfie. The responses to the stress and anxiety questions, along with the corresponding selfie photographs, are then analyzed on an individual basis. This analysis focuses on exploring the relationships between self-reported stress and anxiety levels and potential facial expressions indicative of stress and anxiety, eye features such as pupil size variation and eye closure, and specific action units (AUs) observed in the frames over time. In addition to its primary functions, the mobile app also gathers sensor data, including accelerometer and gyroscope readings, on a daily basis. This data holds potential for further analysis related to stress and anxiety. Furthermore, apart from capturing selfie photographs, participants have the option to upload video recordings of themselves while engaging in two neuropsychological games. These recorded videos are then subjected to analysis in order to extract pertinent features that can be utilized for binary classification of stress and anxiety (i.e., stress and anxiety recognition). The participants that will be selected for this phase are students aged between 18 and 38, who have received recent clinical diagnoses indicating specific stress and anxiety levels. In order to enhance user engagement in the intervention, gamified elements - an emerging trend to influence user behavior and lifestyle - has been utilized. Incorporating gamified elements into non-game contexts (e.g., health-related) has gained overwhelming popularity during the last few years which has made the interventions more delightful, engaging, and motivating. In the subsequent phase of this research, we conducted an AI experiment employing a personalized machine learning approach to perform emotion recognition on an established dataset called Emognition. This experiment served as a simulation of the future analysis that will be conducted as part of a more comprehensive study focusing on stress and anxiety recognition. The outcomes of the emotion recognition experiment in this study highlight the effectiveness of personalized machine learning techniques and bear significance for the development of future diagnostic endeavors. For training purposes, we selected three models, namely KNN, Random Forest, and MLP. The preliminary performance accuracy results for the experiment were 93%, 95%, and 87% respectively for these models

Psychological research in the digital age

The smartphone has become an important personal companion in our daily lives. Each time we use the device, we generate data that provides information about ourselves. This data, in turn, is valuable to science because it objectively reflects our everyday behavior and experiences. In this way, smartphones enable research that is closer to everyday life than traditional laboratory experiments and questionnaire-based methods. While data collected with smartphones are increasingly being used in the field of personality psychology, new digital technologies can also be leveraged to collect and analyze large-scale unobtrusively sensed data in other areas of psychological research. This dissertation, therefore, explores the insights that smartphone sensing reveals for psychological research using two examples, situation and affect research, making a twofold research contribution. First, in two empirical studies, different data types of smartphone-sensed data, such as GPS or phone data, were combined with experience-sampled self-report, and classical questionnaire data to gain valuable insights into individual behavior, thinking, and feeling in everyday life. Second, predictive modeling techniques were applied to analyze the large, high-dimensional data sets collected by smartphones. To gain a deeper understanding of the smartphone data, interpretable variables were extracted from the raw sensing data, and the predictive performance of various machine learning algorithms was compared. In summary, the empirical findings suggest that smartphone data can effectively capture certain situational and behavioral indicators of psychological phenomena in everyday life. However, in certain research areas such as affect research, smartphone data should only complement, but not completely replace, traditional questionnaire-based data as well as other data sources such as neurophysiological indicators. The dissertation also concludes that the use of smartphone sensor data introduces new difficulties and challenges for psychological research and that traditional methods and perspectives are reaching their limits. The complexity of data collection, processing, and analysis requires established guidelines for study design, interdisciplinary collaboration, and theory-driven research that integrates explanatory and predictive approaches. Accordingly, further research is needed on how machine learning models and other big data methods in psychology can be reconciled with traditional theoretical approaches. Only in this way can we move closer to the ultimate goal of psychology to better understand, explain, and predict human behavior and experiences and their interplay with everyday situations

New methods for stress assessment and monitoring at the workplace

The topic of stress is nowadays a very important one, not only in research but on social life in general. People are increasingly aware of this problem and its consequences at several levels: health, social life, work, quality of life, etc. This resulted in a significant increase in the search for devices and applications to measure and manage stress in real-time. Recent technological and scientific evolution fosters this interest with the development of new methods and approaches. In this paper we survey these new methods for stress assessment, focusing especially on those that are suited for the workplace: one of today’s major sources of stress. We contrast them with more traditional methods and compare them between themselves, evaluating nine characteristics. Given the diversity of methods that exist nowadays, this work facilitates the stakeholders’ decision towards which one to use, based on how much their organization values aspects such as privacy, accuracy, cost-effectiveness or intrusivenes

New methods for stress assessment and monitoring at the workplace

The topic of stress is nowadays a very important one, not only in research but on social life in general. People are increasingly aware of this problem and its consequences at several levels: health, social life, work, quality of life, etc. This resulted in a significant increase in the search for devices and applications to measure and manage stress in real-time. Recent technological and scientific evolution fosters this interest with the development of new methods and approaches. In this paper we survey these new methods for stress assessment, focusing especially on those that are suited for the workplace: one of today’s major sources of stress. We contrast them with more traditional methods and compare them between themselves, evaluating nine characteristics. Given the diversity of methods that exist nowadays, this work facilitates the stakeholders’ decision towards which one to use, based on how much their organization values aspects such as privacy, accuracy, cost-effectiveness or intrusivenes