How to Relax in Stressful Situations: A Smart Stress Reduction System

Stress is an inescapable element of the modern age. Instances of untreated stress may lead to a reduction in the individual's health, well-being and socio-economic situation. Stress management application development for wearable smart devices is a growing market. The use of wearable smart devices and biofeedback for individualized real-life stress reduction interventions has received less attention. By using our unobtrusive automatic stress detection system for use with consumer-grade smart bands, we first detected stress levels. When a high stress level is detected, our system suggests the most appropriate relaxation method by analyzing the physical activity-based contextual information. In more restricted contexts, physical activity is lower and mobile relaxation methods might be more appropriate, whereas in free contexts traditional methods might be useful. We further compared traditional and mobile relaxation methods by using our stress level detection system during an eight day EU project training event involving 15 early stage researchers (mean age 28; gender 9 Male, 6 Female). Participants' daily stress levels were monitored and a range of traditional and mobile stress management techniques was applied. On day eight, participants were exposed to a 'stressful' event by being required to give an oral presentation. Insights about the success of both traditional and mobile relaxation methods by using the physiological signals and collected self-reports were provided

Approaches, applications, and challenges in physiological emotion recognition — a tutorial overview

An automatic emotion recognition system can serve as a fundamental framework for various applications in daily life from monitoring emotional well-being to improving the quality of life through better emotion regulation. Understanding the process of emotion manifestation becomes crucial for building emotion recognition systems. An emotional experience results in changes not only in interpersonal behavior but also in physiological responses. Physiological signals are one of the most reliable means for recognizing emotions since individuals cannot consciously manipulate them for a long duration. These signals can be captured by medical-grade wearable devices, as well as commercial smart watches and smart bands. With the shift in research direction from laboratory to unrestricted daily life, commercial devices have been employed ubiquitously. However, this shift has introduced several challenges, such as low data quality, dependency on subjective self-reports, unlimited movement-related changes, and artifacts in physiological signals. This tutorial provides an overview of practical aspects of emotion recognition, such as experiment design, properties of different physiological modalities, existing datasets, suitable machine learning algorithms for physiological data, and several applications. It aims to provide the necessary psychological and physiological backgrounds through various emotion theories and the physiological manifestation of emotions, thereby laying a foundation for emotion recognition. Finally, the tutorial discusses open research directions and possible solutions

Design and Application of Wireless Body Sensors

Hörmann T. Design and Application of Wireless Body Sensors. Bielefeld: Universität Bielefeld; 2019

Analysis and design of individual information systems to support health behavior change

As a wide-ranging socio-technical transformation, the digitalization has significantly influenced the world, bringing opportunities and challenges to our lives. Despite numerous benefits like the possibility to stay connected with people around the world, the increasing dispersion and use of digital technologies and media (DTM) pose risks to individuals’ well-being and health. Rising demands emerging from the digital world have been linked to digital stress, that is, stress directly or indirectly resulting from DTM (Ayyagari et al. 2011; Ragu-Nathan et al. 2008; Tarafdar et al. 2019; Weil and Rosen 1997), potentially intensifying individuals’ overall exposure to stress. Individuals experiencing this adverse consequence of digitalization are at elevated risk of developing severe mental health impairments (Alhassan et al. 2018; Haidt and Allen 2020; Scott et al. 2017), which is why various scholars emphasize that research should place a stronger focus on analyzing and shaping the role of the individual in a digital world, pursuing instrumental as well as humanistic objectives (Ameen et al. 2021; Baskerville 2011b). Information Systems (IS) research has long placed emphasis on the use of information and communication technology (ICT) in organizations, viewing an information system as the socio-technical system that emerges from individuals’ interaction with DTM in organizations. However, socio-technical information systems, as the essence of the IS discipline (Lee 2004; Sarker et al. 2019), are also present in different social contexts from private life. Acknowledging the increasing private use of DTM, such as smartphones and social networks, IS scholars have recently intensified their efforts to understand the human factor of IS (Avison and Fitzgerald 1991; Turel et al. 2021). A framework recently proposed by Matt et al. (2019) suggests three research angles: analyzing individuals’ behavior associated with their DTM use, analyzing what consequences arise from their DTM use behavior, and designing new technologies that promote positive or mitigate negative effects of individuals’ DTM use. Various recent studies suggest that individuals’ behavior seems to be an important lever influencing the outcomes of their DTM use (Salo et al. 2017; Salo et al. 2020; Weinstein et al. 2016). Therefore, this dissertation aims to contribute to IS research targeting the facilitation of a healthy DTM use behavior. It explores the use behavior, consequences, and design of DTM for individuals' use with the objective to deliver humanistic value by increasing individuals' health through supporting a behavior change related to their DTM use. The dissertation combines behavioral science and design science perspectives and applies pluralistic methodological approaches from qualitative (e.g., interviews, prototyping) and quantitative research (e.g., survey research, field studies), including mixed-methods approaches mixing both. Following the framework from Matt et al. (2019), the dissertation takes three perspectives therein: analyzing individuals’ behavior, analyzing individuals’ responses to consequences of DTM use, and designing information systems assisting DTM users. First, the dissertation presents new descriptive knowledge on individuals’ behavior related to their use of DTM. Specifically, it investigates how individuals behave when interacting with DTM, why they behave the way they do, and how their behavior can be influenced. Today, a variety of digital workplace technologies offer employees different ways of pursuing their goals or performing their tasks (Köffer 2015). As a result, individuals exhibit different behaviors when interacting with these technologies. The dissertation analyzes what interactional roles DTM users can take at the digital workplace and what may influence their behavior. It uses a mixed-methods approach and combines a quantitative study building on trace data from a popular digital workplace suite and qualitative interviews with users of this digital workplace suite. The empirical analysis yields eight user roles that advance the understanding of users’ behavior at the digital workplace and first insights into what factors may influence this behavior. A second study adds another perspective and investigates how habitual behavior can be changed by means of DTM design elements. Real-time feedback has been discussed as a promising way to do so (Schibuola et al. 2016; Weinmann et al. 2016). In a field experiment, employees working at the digital workplace are provided with an external display that presents real-time feedback on their office’s indoor environmental quality. The experiment examines if and to what extent the feedback influences their ventilation behavior to understand the effect of feedback as a means of influencing individuals’ behavior. The results suggest that real-time feedback can effectively alter individuals’ behavior, yet the feedback’s effectiveness reduces over time, possibly as a result of habituation to the feedback. Second, the dissertation presents new descriptive and prescriptive knowledge on individuals’ ways to mitigate adverse consequences arising from the digitalization of individuals. A frequently discussed consequence that digitalization has on individuals is digital stress. Although research efforts strive to determine what measures individuals can take to effectively cope with digital stress (Salo et al. 2017; Salo et al. 2020; Weinert 2018), further understanding of individuals’ coping behavior is needed (Weinert 2018). A group at high risk of suffering from the adverse effects of digital stress is adolescents because they grow up using DTM daily and are still developing their identity, acquiring mental strength, and adopting essential social skills. To facilitate a healthy DTM use, the dissertation explores what strategies adolescents use to cope with the demands of their DTM use. Combining a qualitative and a quantitative study, it presents 30 coping responses used by adolescents, develops five factors underlying adolescents’ activation of coping responses, and identifies gender- and age-related differences in their coping behavior. Third, the dissertation presents new prescriptive knowledge on the design of individual information systems supporting individuals in understanding and mitigating their perceived stress. Facilitated by the sensing capabilities of modern mobile devices, it explores the design and development of mobile systems that assess stress and support individuals in coping with stress by initiating a change of stress-related behavior. Since there is currently limited understanding of how to develop such systems, this dissertation explores various facets of their design and development. As a first step, it presents the development of a prototype aiming for life-integrated stress assessment, that is, the mobile sensor-based assessment of an individual’s stress without interfering with their daily routines. Data collected with the prototype yields a stress model relating sensor data to individuals’ perception of stress. To deliver a more generalized perspective on mobile stress assessment, the dissertation further presents a literature- and experience-based design theory comprising a design blueprint, design requirements, design principles, design features, and a discussion of potentially required trade-offs. Mobile stress assessment may be used for the development of mobile coping assistants. Aiming to assist individuals in effectively coping with stress and preventing future stress, a mobile coping assistant should recommend adequate coping strategies to the stressed individual in real-time or execute targeted actions within a defined scope of action automatically. While the implementation of a mobile coping assistant is yet up to future research, the dissertation presents an abstract design and algorithm for selecting appropriate coping strategies. To sum up, this dissertation contributes new knowledge on the digitalization of individuals to the IS knowledge bases, expanding both descriptive and prescriptive knowledge. Through the combination of diverse methodological approaches, it delivers knowledge on individuals’ behavior when using DTM, on the mitigation of consequences that may arise from individuals’ use of DTM, and on the design of individual information systems with the goal of facilitating a behavior change, specifically, regarding individuals’ coping with stress. Overall, the research contained in this dissertation may promote the development of digital assistants that support individuals’ in adopting a healthy DTM use behavior and thereby contribute to shaping a socio-technical environment that creates more benefit than harm for all individuals

The potential of emerging wearable physiological sensing in the space of human-subject studies

PhD ThesisIn recent years, novel sensing means in the form of smartwatches and fitness trackers with integrated sophisticated sensing emerged on the consumer market. While their primary purpose is to provide consumers with an overview of rough-grained health-related metrics, these signals offer to pick up fine-grained changes within the human body. This thesis considers the suitability of these novel wearable sensing devices to be used in affective research. Firstly, and based on the work with concrete state-of-the-art wearables, issues around the access of research-suitable data are discussed. The findings are put in context by examining common wearable device architectures and data access means provided. The discussion concludes with aspects researchers need to consider when seeking data access from state-of-the-art or future wearables. Secondly, two research probes explore the application of four exemplary devices to detect stress and affect in the wild and in the lab. Issues around the data reliability and participant comfort arose. The experiences are reflected upon to provide researchers with a summary of aspects to consider when applying wearable sensing devices in affective research. Lastly, this thesis contributes a Design Space for Physiological Measurement Tools. This design space was evaluated with a qualitative study enquiring research experts experiences. The resulting Design Space presents seven distinct dimensions of factors to consider when choosing a wearable sensing device for research. This design space has been applied to a novel sensing device which was used for a study on interpersonal synchrony. The insights and the ‘Design Space for Physiological Measurement Tools’ provide researchers with a tool to apply when they consider to use wearable physiological sensing devices in research

Wearable devices for remote vital signs monitoring in the outpatient setting: an overview of the field

Early detection of physiological deterioration has been shown to improve patient outcomes. Due to recent improvements in technology, comprehensive outpatient vital signs monitoring is now possible. This is the first review to collate information on all wearable devices on the market for outpatient physiological monitoring. A scoping review was undertaken. The monitors reviewed were limited to those that can function in the outpatient setting with minimal restrictions on the patient’s normal lifestyle, while measuring any or all of the vital signs: heart rate, ECG, oxygen saturation, respiration rate, blood pressure and temperature. A total of 270 papers were included in the review. Thirty wearable monitors were examined: 6 patches, 3 clothing-based monitors, 4 chest straps, 2 upper arm bands and 15 wristbands. The monitoring of vital signs in the outpatient setting is a developing field with differing levels of evidence for each monitor. The most common clinical application was heart rate monitoring. Blood pressure and oxygen saturation measurements were the least common applications. There is a need for clinical validation studies in the outpatient setting to prove the potential of many of the monitors identified. Research in this area is in its infancy. Future research should look at aggregating the results of validity and reliability and patient outcome studies for each monitor and between different devices. This would provide a more holistic overview of the potential for the clinical use of each device

On the prediction of clinical outcomes using Heart Rate Variability estimated from wearable devices

This thesis explores the use of Heart Rate Variability as a tool for predicting health outcomes, focusing on data derived from photoplethysmography (PPG) sensors in wrist-worn wearable devices such as smartwatches. These devices offer a unique opportunity for cost-effective, continuous, and unobtrusive monitoring of heart health. However, PPG data is susceptible to motion artefacts, challenging the reliability of Heart Rate Variability metrics derived from it. A critical finding of this research is the unreliability of specific frequency-domain Heart Rate Variability features, such as the Sympathovagal Balance Index (SVI), due to low signal-to-noise ratio in certain frequency bands. Conversely, the thesis demonstrates that most HRV features, including Root Mean Square of Successive Differences between normal heartbeats (RMSSD) and Standard Deviation of Normal heartbeats (SDNN), can be reliably extracted under conditions of motion, such as during physical activity or recovery from exercise. This is achieved by employing accelerometry data from wearable devices to filter out unreliable PPG data. The thesis also addresses the issue of missing data in Heart Rate Variability analysis, a consequence of motion artefacts and the energy-saving strategies of wearable devices. By exploring different interpolation methods and their effects on Heart Rate Variability features, this research identifies the best approaches for handling missing data. Particularly, it recommends operating on timestamp time-series over duration time-series, contradicting traditional Heart Rate Variability preprocessing practices. Quadratic interpolation in the time domain was identified as the most effective method, introducing minimal error across numerous Heart Rate Variability features, contrary to interpolation in the duration domain. The research presented in this thesis evaluates Heart Rate Variability features derived from ultra-short measurement windows, demonstrating the feasibility of accurately estimating RMSSD and SDNN using 30-second and 1-minute time windows, respectively. This study, unique in assessing the effect of missing values on ultra-short Heart Rate Variability data, reveals that missing values significantly impact SDNN estimations while moderately affecting RMSSD. The analysis highlights that ultra-short inter-beat interval time series limit the assessment of very low frequency (VLF) components, increasing bias in SDNN estimates. This finding is particularly significant in light of the prevalent use of SDNN in commercial wearables, underscoring its importance for continuous heart health monitoring. The study notes that the shorter the measurement window and the greater the amount of missing values, the larger the bias observed in SDNN. A novel aspect of the thesis is the creation of an innovative mathematical model designed to estimate the impact of circadian rhythms on resting heart rate. This model stands out for its computational efficiency, making it particularly suitable for data obtained from wearable devices. It surpasses the single component cosinor model in accuracy, demonstrated by a lower root mean square error (RMSE) in predicting future heart rate values. Additionally, it retains the advantage of providing easily interpretable parameters, such as MESOR, Acrophase, and Amplitude, which are essential for assessing changes in heart activity. The thesis demonstrates that Heart Rate data can accurately estimate SDNN24 (the Standard Deviation of NN intervals over 24 hours), with a difference of about 0.22±11.47 (RMSE = 53.81 and $r^2 = 0.97$). This finding indicates that despite being fragmentary, 24-hour HR data from wrist-worn fitness devices is adequate for estimating SDNN24 and assessing health status, as evidenced by an F1 score of 0.97. The robustness of SDNN24 estimation against noisy data suggests that wrist-worn wearables are capable of reliably monitoring cardiovascular health on a continuous basis, thus facilitating early interventions in response to changes in Sinoatrial Node activity. The final part of the thesis introduces an innovative approach to health outcome prediction, employing Heart Rate Variability data gathered during exercise alongside Electronic Health Record data. Employing Large Language Models to process EHR data and Convolutional AutoEncoders for Heart Rate Variability analysis, this approach reveals the untapped potential of exercise Heart Rate Variability data in health monitoring and prediction. Deep Learning models incorporating Heart Rate Variability data demonstrated enhanced predictive accuracy for cardiovascular diseases (CVD), coronary heart disease (CHD), and Angina, evidenced by higher Area Under the Curve (AUC) scores compared to models using only Electronic Health Records and demographic/behavioural data. The highest AUC scores achieved were 0.71 for CVD, 0.74 for CHD, and 0.73 for Angina. In conclusion, this thesis contributes to the field of biomedical engineering by enhancing the understanding and application of HRV analysis in health outcome prediction using wearable device data. It offers insights for future work in continuous, unobtrusive health monitoring and underscores the need for further research in this rapidly evolving domain

Towards multimodal driver state monitoring systems for highly automated driving

Real-time monitoring of drivers’ functional states will soon become a required safety feature for commercially available vehicles with automated driving capability. Automated driving technology aims to mitigate human error from road transport with the progressive automatisation of specific driving tasks. However, while control of the driving task remains shared between humans and automated systems, the inclusion of this new technology is not exempt from other human factors-related challenges. Drivers’ functional states are essentially a combination of psychological, emotional, and cognitive states, and they generate a constant activity footprint available for measurement through neural and peripheral physiology, among other measures. These factors can determine drivers’ functional states and, thus, drivers’ availability to safely perform control transitions between human and vehicle. This doctoral project aims at investigating the potential of electrocardiogram (ECG), electrodermal activity (EDA) and functional near-infrared spectroscopy (fNIRS) as measures for a multimodal driver state monitoring (DSM) system for highly automated driving (i.e., SAE levels 3 and 4). While current DSM systems relying on gaze behaviour measures have proven valid and effective, several limitations and challenges could only be overcome using eye-tracking in tandem with physiological parameters. This thesis investigates whether ECG, EDA and fNIRS would be good candidates for such a purpose. Two driving simulator studies were performed to measure mental workload, trust in automation, stress and perceived risk, all identified as modulators of drivers’ functional states and that could eventually determine drivers’ availability to take-over manual control. The main findings demonstrate that DSM systems should adopt multiple physiological measures to capture changes in functional states relevant for driver readiness. Future DSM systems will benefit from the knowledge generated by this research by applying machine learning methods to these measures for determining drivers’ availability for optimal take-over performance