Emotions in context: examining pervasive affective sensing systems, applications, and analyses

Pervasive sensing has opened up new opportunities for measuring our feelings and understanding our behavior by monitoring our affective states while mobile. This review paper surveys pervasive affect sensing by examining and considering three major elements of affective pervasive systems, namely; “sensing”, “analysis”, and “application”. Sensing investigates the different sensing modalities that are used in existing real-time affective applications, Analysis explores different approaches to emotion recognition and visualization based on different types of collected data, and Application investigates different leading areas of affective applications. For each of the three aspects, the paper includes an extensive survey of the literature and finally outlines some of challenges and future research opportunities of affective sensing in the context of pervasive computing

Toward Data-Driven Digital Therapeutics Analytics: Literature Review and Research Directions

With the advent of Digital Therapeutics (DTx), the development of software as a medical device (SaMD) for mobile and wearable devices has gained significant attention in recent years. Existing DTx evaluations, such as randomized clinical trials, mostly focus on verifying the effectiveness of DTx products. To acquire a deeper understanding of DTx engagement and behavioral adherence, beyond efficacy, a large amount of contextual and interaction data from mobile and wearable devices during field deployment would be required for analysis. In this work, the overall flow of the data-driven DTx analytics is reviewed to help researchers and practitioners to explore DTx datasets, to investigate contextual patterns associated with DTx usage, and to establish the (causal) relationship of DTx engagement and behavioral adherence. This review of the key components of data-driven analytics provides novel research directions in the analysis of mobile sensor and interaction datasets, which helps to iteratively improve the receptivity of existing DTx.Comment: This paper has been accepted by the IEEE/CAA Journal of Automatica Sinic

Innovative methods for observing and changing complex health behaviors: Four propositions

Precision health initiatives aim to progressively move from traditional, group-level approaches to health diagnostics and treatments toward ones that are individualized, contextualized, and timely. This article aims to provide an overview of key methods and approaches that can help facilitate this transition in the health behavior change domain. This article is a narrative review of the methods used to observe and change complex health behaviors. On the basis of the available literature, we argue that health behavior change researchers should progressively transition from (i) low- to high-resolution behavioral assessments, (ii) group-only to group- and individual-level statistical inference, (iii) narrative theoretical models to dynamic computational models, and (iv) static to adaptive and continuous tuning interventions. Rather than providing an exhaustive and technical presentation of each method and approach, this article articulates why and how researchers interested in health behavior change can apply these innovative methods. Practical examples contributing to these efforts are presented. If successfully adopted and implemented, the four propositions in this article have the potential to greatly improve our public health and behavior change practices in the near future

Advancing Models and Theories for Digital Behavior Change Interventions

To be suitable for informing digital behavior change interventions, theories and models of behavior change need to capture individual variation and changes over time. The aim of this paper is to provide recommendations for development of models and theories that are informed by, and can inform, digital behavior change interventions based on discussions by international experts, including behavioral, computer, and health scientists and engineers. The proposed framework stipulates the use of a state-space representation to define when, where, for whom, and in what state for that person, an intervention will produce a targeted effect. The "state" is that of the individual based on multiple variables that define the "space" when a mechanism of action may produce the effect. A state-space representation can be used to help guide theorizing and identify crossdisciplinary methodologic strategies for improving measurement, experimental design, and analysis that can feasibly match the complexity of real-world behavior change via digital behavior change interventions

Using Mobile Apps to Support the Implementation of Coping-relevant Behaviour Change Techniques for Self-management of Stress

Mobile apps have shown potential in early stress self-management interventions, yet they remain less beneficial than face-to-face therapies. One of the most effective ways people can cope with stress is to identify what their stressors are and take action in managing them. Coping-relevant behaviour change techniques (BCTs), such as self-monitoring, goal setting, and action planning, have the potential to support this process. Nevertheless, there is little guidance on how to incorporate such techniques into stress management apps. Drawing on mixed methods research, this thesis provides two contributions. First, it improves our understanding of how existing stress management apps support coping-relevant BCTs and suggests areas for improvements. An app functionality review and follow-up 3-week intervention using Welltory stress monitoring and Coach.me goal setting apps revealed that existing apps do not support users’ efforts with coping-relevant BCTs. Participants reported that Welltory did not yield sufficient data to gain insights into the factors affecting their stress. Relatedly, the way in which these apps implemented coping-relevant BCTs diminished peoples’ sense of autonomy and competence. Drawing on peoples’ experiences with existing apps and principles of positive computing, the second contribution of this thesis is the design and evaluation of Reffy - a chatbot prototype that integrates coping-relevant BCTs in a way that meets people’s stress management needs. Based on findings from a field evaluation study, we identify specific benefits and challenges of using a stress self-management chatbot. We find that chatbot-based reflective questioning helps people identify how factors impact their stress during early stages of self-tracking. Likewise, adding features that promote users’ sense of autonomy and competence improves Welltory’s ability to support coping strategies. This thesis advances our understanding of how behaviour change and stress coping techniques can be incorporated into mobile apps to effectively support stress self-management

Engagement With a Behavior Change App for Alcohol Reduction: Data Visualization for Longitudinal Observational Study

BACKGROUND: Behavior change apps can develop iteratively, where the app evolves into a complex, dynamic, or personalized intervention through cycles of research, development, and implementation. Understanding how existing users engage with an app (eg, frequency, amount, depth, and duration of use) can help guide further incremental improvements. We aim to explore how simple visualizations can provide a good understanding of temporal patterns of engagement, as usage data are often longitudinal and rich. OBJECTIVE: This study aims to visualize behavioral engagement with Drink Less, a behavior change app to help reduce hazardous and harmful alcohol consumption in the general adult population of the United Kingdom. METHODS: We explored behavioral engagement among 19,233 existing users of Drink Less. Users were included in the sample if they were from the United Kingdom; were 18 years or older; were interested in reducing their alcohol consumption; had a baseline Alcohol Use Disorders Identification Test score of 8 or above, indicative of excessive drinking; and had downloaded the app between May 17, 2017, and January 22, 2019 (615 days). Measures of when sessions begin, length of sessions, time to disengagement, and patterns of use were visualized with heat maps, timeline plots, k-modes clustering analyses, and Kaplan-Meier plots. RESULTS: The daily 11 AM notification is strongly associated with a change in engagement in the following hour; reduction in behavioral engagement over time, with 50.00% (9617/19,233) of users disengaging (defined as no use for 7 or more consecutive days) 22 days after download; identification of 3 distinct trajectories of use, namely engagers (4651/19,233, 24.18% of users), slow disengagers (3679/19,233, 19.13% of users), and fast disengagers (10,903/19,233, 56.68% of users); and limited depth of engagement with 85.076% (7,095,348/8,340,005) of screen views occurring within the Self-monitoring and Feedback module. In addition, a peak of both frequency and amount of time spent per session was observed in the evenings. CONCLUSIONS: Visualizations play an important role in understanding engagement with behavior change apps. Here, we discuss how simple visualizations helped identify important patterns of engagement with Drink Less. Our visualizations of behavioral engagement suggest that the daily notification substantially impacts engagement. Furthermore, the visualizations suggest that a fixed notification policy can be effective for maintaining engagement for some users but ineffective for others. We conclude that optimizing the notification policy to target both effectiveness and engagement is a worthwhile investment. Our future goal is to both understand the causal effect of the notification on engagement and further optimize the notification policy within Drink Less by tailoring to contextual circumstances of individuals over time. Such tailoring will be informed from the findings of our micro-randomized trial (MRT), and these visualizations were useful in both gaining a better understanding of engagement and designing the MRT

Detecting Stressful Social Interactions Using Wearable Physiological and Inertial Sensors

Stress is unavoidable in everyday life which can result in several health related short and long-term adverse consequences. Previous research found that most of the stress events occur due to interpersonal tension followed by work related stress. Enabling automated detection of stressful social interactions using wearable technology will help trigger just-in-time interventions which can help the user cope with the stressful situation. In this dissertation, we show the feasibility of differentiating stressful social interactions from other stressors i.e., work and commute.However, collecting reliable ground truth stressor data in the natural environment is challenging. This dissertation addresses this challenge by designing a Day Reconstruction Method (DRM) based contextual stress visualization that highlights the continuous stress inferences from a wearable sensor with surrounding activities such as conversation, physical activity, and location on a timeline diagram. This dissertation proposes a Conditional Random Field, Context-Free Grammar (CRF-CFG) model to detect conversation from breathing patterns to support the visualization. The advantage of breathing signal is that it does not capture the content of the conversation and hence, is more privacy preserving compared to audio. It proposes a framework to systematically analyze the breathing data collected in the natural environment. However, it requires wearing of chest worn sensor. This dissertation aims to determine stressful social interaction without wearing chest worn sensor or without requiring any conversation model which is privacy sensitive. Therefore, it focuses on detecting stressful social interactions directly from stress time-series only which can be captured using increasingly available wrist worn sensor.This dissertation proposes a framework to systematically analyze the respiration data collected in the natural environment. The analysis includes screening the low-quality data, segmenting the respiration time-series by cycles, and develop time-domain features. It proposes a Conditional Random Field, Context-Free Grammar (CRF-CFG) model to detect conversation episodes from breathing patterns. This system is validated against audio ground-truth in the field with an accuracy of 71.7\%.This dissertation introduces the stress cycle concept to capture the cyclical patterns and identifies novel features from stress time-series data. Furthermore, wrist-worn accelerometry data shows that hand gestures have a distinct pattern during stressful social interactions. The model presented in this dissertation augments accelerometry patterns with the stress cycle patterns for more accurate detection. Finally, the model is trained and validated using data collected from 38 participants in free-living conditions. The model can detect the stressful interactions with an F1-score of 0.83 using stress cycle features and enable the delivery of stress intervention within 3.9 minutes since the onset of a stressful social interaction

ViBreathe: Heart Rate Variability Enhanced Respiration Training for Workaday Stress Management via an Eyes-free Tangible Interface

Slow breathing guiding applications increasingly emerge, showing promise for helping knowledge workers to better cope with workaday stress. However, standard breathing guidance is non-interactive, with rigid paces. Despite their effects being proved, they could cause respiratory fatigue, or lack of training motivation, especially for novice users. To explore new design possibilities, we investigate using heart rate variability (HRV) data to mediate breathing guidance, which results in two HRV-enhanced guidance modes: (i) responsive breathing guidance and (ii) adaptive breathing guidance. These guidance modes are implemented on a soft haptic interface named “ViBreathe”. We conducted a user test (N\ua0=\ua024), and a one-week field deployment (N\ua0=\ua04) with knowledge workers, to understand the user experience of our design. The HRV-enhanced modes were generally experienced to reduce tiresome and improve engagement and comfort. And Vibreathe showed great potential for seamlessly weaving slow breathing practice into work routines. We thereby summarize related design insights and opportunities

Supporting Diabetes Self-Management with Ubiquitous Computing Technologies: A User-Centered Inquiry

Ubiquitous computing technologies offer opportunities to improve treatments for chronic health conditions. Type 1 diabetes is a compelling use-case for such approaches, given its severity, and need for individuals to make frequent care decisions, informed by complex data. However, current apps, typically based on effortful reflection on collected data, generally show poor adoption, lack vital cognitive and emotional support, and are poorly tailored to users’ actual diabetes decision making processes. This thesis investigates how diabetes apps can be improved from a user-centered perspective. An initial questionnaire-based study investigated how well existing diabetes apps meet user needs. Perceived benefits, limitations, and reasons for low adoption rates were identified. A talk-aloud study of detailed user interactions with diabetes logging apps was conducted to characterize the benefits and limitations of diverse UI elements for T1 diabetes management, and to more precisely identify wider problems with current interaction designs. This led to positing a refined version of Mamykina et al.’s model for diabetes self-management, to account for observed practices, whereby the previously accepted habitual and sensemaking cognitive states are augmented by a posited ‘fluid contextual reasoning’ (FCR) mode, which allows multiple contextual factors to be balanced for dynamic course correction when navigating complex situations, using previously learned knowledge. To investigate user perceptions of the levels and kinds of monitoring anticipated in next generation diabetes decision support systems, a 4-week technology probe, in which participants used multiple networked devices and external data aggregation, was used to frame requirements for user-centered development of such future systems. Integrating all of the above work, an iterative design process was undertaken to create DUETS, a card-based system to facilitate reflection by designers, users, and other stakeholders on diabetes support management systems. The resulting tool and method were then implemented and evaluated through structured sessions with stakeholder focus groups