Use of a non-human robot audience to induce stress reactivity in human participants

This study examined whether a non-human robot audience can elicit a stress response in human participants. A 90-minute experimental laboratory session based on the Trier Social Stress Test (TSST) using a pre-recorded robot audience, was presented as a live on-screen simulation. Nineteen participants (female=16) aged 21–57 years (M=29.74) underwent a ten-minute mock interview and mathematics task in front of the robot audience. Salivary cortisol was assessed at 10-minutes before and immediately prior to the start of the stress test, and +10-, +30- and +40-minutes after the start of the test. Heart rate was assessed 20 minutes before, at five minutes into and 40-minutes after the test. Perceived stress and trait coping responses were provided at entry and participants were interviewed post task about their subjective experience. Significant increases in salivary cortisol and heart rate were observed over time with no significant interactions by participant subjective report. Coping responses including active coping and planning showed significant relationships with cortisol and heart rate reactivity and recovery. Until now, a non-human robot audience has not been used in a social stress testing paradigm. This methodology offers an innovative application with potential for further in-depth evaluation of stress reactivity and adaptation

Ein innovativer Ansatz für die Induktion und Messung von Akuten Stressreaktionen: Entwicklung, Evaluierung und Anwendungspotential eines Digitalen Stress Tests

Acute stress is linked to a variety of negative outcomes, including increased risk for mental and physical diseases, and reduced quality of life. Effective induction and accurate measurement of acute stress responses are important for both research and clinical purposes. Traditional methods rely on laboratory-based stressors, which can be costly, time-consuming, and impractical for large-scale studies or real-world applications. Measurements in outside-the-lab settings mostly reflect subjective stress levels while objective and feasible measures of biological stress consequences are scarce. This thesis aims to overcome these limitations by linking traditional psycholog-ical stress research with innovative computer science methods. First, covered by a published study, the concept, development and online evaluation of a new Digital Stress Test (DST) for the induction and video-recording of acute stress responses are presented. In this study, the first prototype of the DST was tested in a large and experimenter-independent online study with 284 participants. Results show that the DST could induce significantly higher levels of perceived stress and negative affect compared to the control condition. Going beyond this study, further developments of the DST and a pre-registered follow-up validation study are outlined. In this study, participants perform the DST and the gold standard laboratory stress induction paradigm Trier Social Stress Test while their physiological stress responses are evaluated. Lastly, the potentials of using the DST to contribute to the development of video-based stress detection methods are critically reviewed. Therefore, a follow-up online study for collecting a video dataset is outlined and, based on the results of a further already published study, the applicability of baseline machine learning algorithms for video-based stress detection discussed. The findings in this thesis imply several potentials of the Digital Stress Test: First, the DST is applicable as a tool for inducing acute stress responses in outside-the-lab settings and thus making more ecologically valid and scalable stress studies possible. Secondly, it also allows for gathering videos capturing stress-related behavioral data in real-world scenarios and therefore supporting the development of reliable stress detection algorithms. Finally, this thesis may present the DST as an invitation for promoting open and collaborative research in the interdisciplinary field between psychology and computer science.Akuter Stress ist mit einer Vielzahl negativer Auswirkungen verbunden, einschließlich einem erhöhtem Risiko für psychische und körperliche Erkrankungen sowie reduzierter Lebensqualität. Eine wirksame Induktion und genaue Messung akuter Stressreaktionen ist sowohl für Forschungs- als auch für klinische Zwecke relevant. Traditionelle Methoden setzen auf im Labor durchgeführte Stressoren, die kostenintensiv, zeitaufwendig und unpraktisch für groß angelegte Studien oder Anwendungen im alltäglichen Leben sein können. Messungen außerhalb des Labors spiegeln meist das subjektive Stresslevel wider, während objektive und alltagstaugliche Methoden zur Messung von biologischen Stressfolgen fehlen. Diese Dissertation zielt darauf ab, diese Einschränkungen durch die Verbindung traditioneller psychologischer Stressforschung mit innovativen Methoden der Informatik zu überwinden. Zunächst wird die veröffentlichte Studie über das Konzept, die Entwicklung und die Online-Evaluation eines neuen Digitalen Stress Tests (DST) für die Induktion und Videoaufzeichnung akuter Stressreaktionen vorgestellt. In dieser Studie wurde der erste Prototyp des DST in einer großen und experimentatorunabhängigen Online-Studie mit 284 Teilnehmenden getestet und konnte im Vergleich zur Kontrollbedingung signifikant stärkeren wahrgenommenen Stress und negativen Affekt auslösen. Über die Studie hinausgehend werden Weiterentwicklungen des DST und eine prä-registrierte Validierungsstudie skizziert. In dieser zusätzlichen Studie führen die Teilnehmenden den DST und das Goldstandard-Stressinduktionsparadigma Trier Social Stress Test durch, wobei Daten zu physiologischen Stressreaktionen erhoben werden. Abschließend wird das Potential, den DST für die Entwicklung von videobasierten Stresserkennungsalgorithmen zu nutzen, kritisch überprüft. Dafür werden Pläne einer weiteren Online-Studie zur Erstellung eines Videodatensatzes skizziert und, basierend auf den Ergebnissen einer weiteren bereits veröffentlichen Studie, die Anwendbarkeit von Grundlagenalgorithmen des maschinellen Lernens für die videobasierte Stresserkennung diskutiert. Die Ergebnisse dieser Dissertation zeigen die vielfältigen Einsatzmöglichkeiten des DST auf: Zunächst kann der DST zur Induktion akuter Stressreaktionen außerhalb des Labors angewendet werden und somit ökologisch valide und skalierbare Stressstudien ermöglichen. Darüber hinaus ermöglicht er die Sammlung von Videos, die stressbezogene Verhaltensdaten in realen Szenarien erfassen und unterstützt damit die Entwicklung von zuverlässigeren Stress-Detektionsalgorithmen. Zusammenfassend können diese Dissertation und der DST als Einladung zur Förderung offener und kollaborativer Forschung im interdisziplinären Bereich zwischen Psychologie und Informatik dienen

INNOVATING CONTROL AND EMOTIONAL EXPRESSIVE MODALITIES OF USER INTERFACES FOR PEOPLE WITH LOCKED-IN SYNDROME

Patients with Lock-In-Syndrome (LIS) lost their ability to control any body part beside their eyes. Current solutions mainly use eye-tracking cameras to track patients' gaze as system input. However, despite the fact that interface design greatly impacts user experience, only a few guidelines have been were proposed so far to insure an easy, quick, fluid and non-tiresome computer system for these patients. On the other hand, the emergence of dedicated computer software has been greatly increasing the patients' capabilities, but there is still a great need for improvements as existing systems still present low usability and limited capabilities. Most interfaces designed for LIS patients aim at providing internet browsing or communication abilities. State of the art augmentative and alternative communication systems mainly focus on sentences communication without considering the need for emotional expression inextricable from human communication. This thesis aims at exploring new system control and expressive modalities for people with LIS. Firstly, existing gaze-based web-browsing interfaces were investigated. Page analysis and high mental workload appeared as recurring issues with common systems. To address this issue, a novel user interface was designed and evaluated against a commercial system. The results suggested that it is easier to learn and to use, quicker, more satisfying, less frustrating, less tiring and less prone to error. Mental workload was greatly diminished with this system. Other types of system control for LIS patients were then investigated. It was found that galvanic skin response may be used as system input and that stress related bio-feedback helped lowering mental workload during stressful tasks. Improving communication was one of the main goal of this research and in particular emotional communication. A system including a gaze-controlled emotional voice synthesis and a personal emotional avatar was developed with this purpose. Assessment of the proposed system highlighted the enhanced capability to have dialogs more similar to normal ones, to express and to identify emotions. Enabling emotion communication in parallel to sentences was found to help with the conversation. Automatic emotion detection seemed to be the next step toward improving emotional communication. Several studies established that physiological signals relate to emotions. The ability to use physiological signals sensors with LIS patients and their non-invasiveness made them an ideal candidate for this study. One of the main difficulties of emotion detection is the collection of high intensity affect-related data. Studies in this field are currently mostly limited to laboratory investigations, using laboratory-induced emotions, and are rarely adapted for real-life applications. A virtual reality emotion elicitation technique based on appraisal theories was proposed here in order to study physiological signals of high intensity emotions in a real-life-like environment. While this solution successfully elicited positive and negative emotions, it did not elicit the desired emotions for all subject and was therefore, not appropriate for the goals of this research. Collecting emotions in the wild appeared as the best methodology toward emotion detection for real-life applications. The state of the art in the field was therefore reviewed and assessed using a specifically designed method for evaluating datasets collected for emotion recognition in real-life applications. The proposed evaluation method provides guidelines for future researcher in the field. Based on the research findings, a mobile application was developed for physiological and emotional data collection in the wild. Based on appraisal theory, this application provides guidance to users to provide valuable emotion labelling and help them differentiate moods from emotions. A sample dataset collected using this application was compared to one collected using a paper-based preliminary study. The dataset collected using the mobile application was found to provide a more valuable dataset with data consistent with literature. This mobile application was used to create an open-source affect-related physiological signals database. While the path toward emotion detection usable in real-life application is still long, we hope that the tools provided to the research community will represent a step toward achieving this goal in the future. Automatically detecting emotion could not only be used for LIS patients to communicate but also for total-LIS patients who have lost their ability to move their eyes. Indeed, giving the ability to family and caregiver to visualize and therefore understand the patients' emotional state could greatly improve their quality of life. This research provided tools to LIS patients and the scientific community to improve augmentative and alternative communication, technologies with better interfaces, emotion expression capabilities and real-life emotion detection. Emotion recognition methods for real-life applications could not only enhance health care but also robotics, domotics and many other fields of study. A complete system fully gaze-controlled was made available open-source with all the developed solutions for LIS patients. This is expected to enhance their daily lives by improving their communication and by facilitating the development of novel assistive systems capabilities