Logging Stress and Anxiety Using a Gamified Mobile-based EMA Application, and Emotion Recognition Using a Personalized Machine Learning Approach

According to American Psychological Association (APA) more than 9 in 10 (94 percent) adults believe that stress can contribute to the development of major health problems, such as heart disease, depression, and obesity. Due to the subjective nature of stress, and anxiety, it has been demanding to measure these psychological issues accurately by only relying on objective means. In recent years, researchers have increasingly utilized computer vision techniques and machine learning algorithms to develop scalable and accessible solutions for remote mental health monitoring via web and mobile applications. To further enhance accuracy in the field of digital health and precision diagnostics, there is a need for personalized machine-learning approaches that focus on recognizing mental states based on individual characteristics, rather than relying solely on general-purpose solutions. This thesis focuses on conducting experiments aimed at recognizing and assessing levels of stress and anxiety in participants. In the initial phase of the study, a mobile application with broad applicability (compatible with both Android and iPhone platforms) is introduced (we called it STAND). This application serves the purpose of Ecological Momentary Assessment (EMA). Participants receive daily notifications through this smartphone-based app, which redirects them to a screen consisting of three components. These components include a question that prompts participants to indicate their current levels of stress and anxiety, a rating scale ranging from 1 to 10 for quantifying their response, and the ability to capture a selfie. The responses to the stress and anxiety questions, along with the corresponding selfie photographs, are then analyzed on an individual basis. This analysis focuses on exploring the relationships between self-reported stress and anxiety levels and potential facial expressions indicative of stress and anxiety, eye features such as pupil size variation and eye closure, and specific action units (AUs) observed in the frames over time. In addition to its primary functions, the mobile app also gathers sensor data, including accelerometer and gyroscope readings, on a daily basis. This data holds potential for further analysis related to stress and anxiety. Furthermore, apart from capturing selfie photographs, participants have the option to upload video recordings of themselves while engaging in two neuropsychological games. These recorded videos are then subjected to analysis in order to extract pertinent features that can be utilized for binary classification of stress and anxiety (i.e., stress and anxiety recognition). The participants that will be selected for this phase are students aged between 18 and 38, who have received recent clinical diagnoses indicating specific stress and anxiety levels. In order to enhance user engagement in the intervention, gamified elements - an emerging trend to influence user behavior and lifestyle - has been utilized. Incorporating gamified elements into non-game contexts (e.g., health-related) has gained overwhelming popularity during the last few years which has made the interventions more delightful, engaging, and motivating. In the subsequent phase of this research, we conducted an AI experiment employing a personalized machine learning approach to perform emotion recognition on an established dataset called Emognition. This experiment served as a simulation of the future analysis that will be conducted as part of a more comprehensive study focusing on stress and anxiety recognition. The outcomes of the emotion recognition experiment in this study highlight the effectiveness of personalized machine learning techniques and bear significance for the development of future diagnostic endeavors. For training purposes, we selected three models, namely KNN, Random Forest, and MLP. The preliminary performance accuracy results for the experiment were 93%, 95%, and 87% respectively for these models

White Paper: Open Digital Health – accelerating transparent and scalable health promotion and treatment

In this White Paper, we outline recommendations from the perspective of health psychology and behavioural science, addressing three research gaps: (1) What methods in the health psychology research toolkit can be best used for developing and evaluating digital health tools? (2) What are the most feasible strategies to reuse digital health tools across populations and settings? (3) What are the main advantages and challenges of sharing (openly publishing) data, code, intervention content and design features of digital health tools? We provide actionable suggestions for researchers joining the continuously growing Open Digital Health movement, poised to revolutionise health psychology research and practice in the coming years. This White Paper is positioned in the current context of the COVID-19 pandemic, exploring how digital health tools have rapidly gained popularity in 2020–2022, when world-wide health promotion and treatment efforts rapidly shifted from face-to-face to remote delivery. This statement is written by the Directors of the not-for-profit Open Digital Health initiative (n = 6), Experts attending the European Health Psychology Society Synergy Expert Meeting (n = 17), and the initiative consultant, following a two-day meeting (19–20th August 2021)

White Paper: Open Digital Health – accelerating transparent and scalable health promotion and treatment

In this White Paper, we outline recommendations from the perspective of health psychology and behavioural science, addressing three research gaps: (1) What methods in the health psychology research toolkit can be best used for developing and evaluating digital health tools? (2) What are the most feasible strategies to reuse digital health tools across populations and settings? (3) What are the main advantages and challenges of sharing (openly publishing) data, code, intervention content and design features of digital health tools? We provide actionable suggestions for researchers joining the continuously growing Open Digital Health movement, poised to revolutionise health psychology research and practice in the coming years. This White Paper is positioned in the current context of the COVID-19 pandemic, exploring how digital health tools have rapidly gained popularity in 2020-2022, when world-wide health promotion and treatment efforts rapidly shifted from face-to-face to remote delivery. This statement is written by the Directors of the not-for-profit Open Digital Health initiative (n = 6), Experts attending the European Health Psychology Society Synergy Expert Meeting (n = 17), and the initiative consultant, following a two-day meeting (19-20th August 2021).Peer reviewe

Internet of Things enabled sedentary behaviour change in office workers: development and feasibility of a novel intervention (WorkMyWay)

Sedentary behaviour (SB) without breaks is associated with adverse health outcomes. The prevalence of prolonged sitting at work among office workers makes a case for SB interventions to target this setting and population. Everyday mundane objects augmented with microelectronics and ubiquitous computing represent a novel mode of delivery for behaviour change interventions enabled by the Internet of Things (IoT). However, there is insufficient research to guide the design of interventions delivered with smart objects. This research addresses this gap by developing WorkMyWay, a workplace SB intervention delivered with IoT-enabled office objects (e.g. smart water bottles and cups), and evaluating its feasibility and acceptability in an 8-week “in-the-wild” study. This thesis made 4 contributions across the disciplines of behavioural medicine and human-computer interactions (HCI). The first contribution is the development of the WorkMyWay intervention, which is informed by findings from a systematic scoping review of prior research in this field (Chapter 3), a diary-probed interview study with 20 office workers (Chapter 4), and a series of technology audit, prototyping, human-centred design, and requirement engineering processes (Chapter 5). Findings from the feasibility study (Chapter 6) suggest that despite technical issues with the data connection, participants perceive high value of WorkMyWay in changing their SB. The intervention is potentially implementable in office-based workplaces, as long as connectivity issues are fixed. Recommendations are made on improvements and a series of future studies in accordance with the Medical Research Council’s guidance on complex intervention development and evaluation. Second, this thesis deepens the theoretical understanding of SB change, by following the Behaviour Change Wheel framework (including the COM-B model, theoretical domain framework, and taxonomies of Behaviour Change Techniques (BCT)) throughout intervention design and evaluation. The intervention contents are specified using the BCT taxonomies (Chapter 5) and informed by the first published COM-B analysis of office worker’s prolonged sitting behaviour at work (Chapter 4). This allows the feasibility study (Chapter 6) to contribute to theory development by matching the interview questions and psychological measures (e.g. strength of habit) with the BCTs (e.g. action planning, prompts and cues) and associated theoretical underpinnings (e.g. goal accessibility). It also allows implementation issues to be considered in light of how well those theories and theory-informed BCTs can work in real-life settings. Third, this thesis makes a methodological contribution by documenting an interdisciplinary approach to develop a digital behaviour change intervention and a model for applying and developing theories of behaviour change in the wild. This helps address the challenge identified in Chapter 3, by bridging the gap between HCI and behavioural medicine, and catalyse the process of feeding technological innovations downstream to health practice and intervention research. Fourth, this research contributes to the HCI literature by proposing a 2×2 matrix framework to guide the design of technology for sustainable behaviour change. On one hand, the framework unifies some of the existing visions and concepts about ubiquitous computing and applies them to the context of behaviour change, by considering the type of cognitive process (automatic versus reflective, based on the dual process model) through which a persuasive design influences the behaviour. For another, the framework considers the required dosage of their technology intervention to maintain the behaviour, or the distribution of changes between the physical world and the human cognition

Internet of Things enabled sedentary behaviour change in office workers: development and feasibility of a novel intervention (WorkMyWay)

Sedentary behaviour (SB) without breaks is associated with adverse health outcomes. The prevalence of prolonged sitting at work among office workers makes a case for SB interventions to target this setting and population. Everyday mundane objects augmented with microelectronics and ubiquitous computing represent a novel mode of delivery for behaviour change interventions enabled by the Internet of Things (IoT). However, there is insufficient research to guide the design of interventions delivered with smart objects. This research addresses this gap by developing WorkMyWay, a workplace SB intervention delivered with IoT-enabled office objects (e.g. smart water bottles and cups), and evaluating its feasibility and acceptability in an 8-week “in-the-wild” study. This thesis made 4 contributions across the disciplines of behavioural medicine and human-computer interactions (HCI). The first contribution is the development of the WorkMyWay intervention, which is informed by findings from a systematic scoping review of prior research in this field (Chapter 3), a diary-probed interview study with 20 office workers (Chapter 4), and a series of technology audit, prototyping, human-centred design, and requirement engineering processes (Chapter 5). Findings from the feasibility study (Chapter 6) suggest that despite technical issues with the data connection, participants perceive high value of WorkMyWay in changing their SB. The intervention is potentially implementable in office-based workplaces, as long as connectivity issues are fixed. Recommendations are made on improvements and a series of future studies in accordance with the Medical Research Council’s guidance on complex intervention development and evaluation. Second, this thesis deepens the theoretical understanding of SB change, by following the Behaviour Change Wheel framework (including the COM-B model, theoretical domain framework, and taxonomies of Behaviour Change Techniques (BCT)) throughout intervention design and evaluation. The intervention contents are specified using the BCT taxonomies (Chapter 5) and informed by the first published COM-B analysis of office worker’s prolonged sitting behaviour at work (Chapter 4). This allows the feasibility study (Chapter 6) to contribute to theory development by matching the interview questions and psychological measures (e.g. strength of habit) with the BCTs (e.g. action planning, prompts and cues) and associated theoretical underpinnings (e.g. goal accessibility). It also allows implementation issues to be considered in light of how well those theories and theory-informed BCTs can work in real-life settings. Third, this thesis makes a methodological contribution by documenting an interdisciplinary approach to develop a digital behaviour change intervention and a model for applying and developing theories of behaviour change in the wild. This helps address the challenge identified in Chapter 3, by bridging the gap between HCI and behavioural medicine, and catalyse the process of feeding technological innovations downstream to health practice and intervention research. Fourth, this research contributes to the HCI literature by proposing a 2×2 matrix framework to guide the design of technology for sustainable behaviour change. On one hand, the framework unifies some of the existing visions and concepts about ubiquitous computing and applies them to the context of behaviour change, by considering the type of cognitive process (automatic versus reflective, based on the dual process model) through which a persuasive design influences the behaviour. For another, the framework considers the required dosage of their technology intervention to maintain the behaviour, or the distribution of changes between the physical world and the human cognition

White Paper: Open Digital Health - accelerating transparent and scalable health promotion and treatment

In this White Paper, we outline recommendations from the perspective of health psychology and behavioural science, addressing three research gaps: (1) What methods in the health psychology research toolkit can be best used for developing and evaluating digital health tools? (2) What are the most feasible strategies to reuse digital health tools across populations and settings? (3) What are the main advantages and challenges of sharing (openly publishing) data, code, intervention content and design features of digital health tools? We provide actionable suggestions for researchers joining the continuously growing Open Digital Health movement, poised to revolutionise health psychology research and practice in the coming years. This White Paper is positioned in the current context of the COVID-19 pandemic, exploring how digital health tools have rapidly gained popularity in 2020-2022, when world-wide health promotion and treatment efforts rapidly shifted from face-to-face to remote delivery. This statement is written by the Directors of the not-for-profit Open Digital Health Initiative (n = 6), Experts attending the European Health Psychology Society Synergy Expert Meeting (n = 17), and the initiative consultant following a two-day meeting (19-20th August 2021).Output Status: Forthcoming/Available Online Additional co-authors: Judith Nalukwago, Efrat Neter, Johanna Nurmi, Manuel Spitschan, Samantha B. Van Beurden, L. Nynke Van der Laan, Kathrin Wunsch, Jasper J. J. Levink & Robbert Sanderma

Quantifying Quality of Life

Describes technological methods and tools for objective and quantitative assessment of QoL Appraises technology-enabled methods for incorporating QoL measurements in medicine Highlights the success factors for adoption and scaling of technology-enabled methods This open access book presents the rise of technology-enabled methods and tools for objective, quantitative assessment of Quality of Life (QoL), while following the WHOQOL model. It is an in-depth resource describing and examining state-of-the-art, minimally obtrusive, ubiquitous technologies. Highlighting the required factors for adoption and scaling of technology-enabled methods and tools for QoL assessment, it also describes how these technologies can be leveraged for behavior change, disease prevention, health management and long-term QoL enhancement in populations at large. Quantifying Quality of Life: Incorporating Daily Life into Medicine fills a gap in the field of QoL by providing assessment methods, techniques and tools. These assessments differ from the current methods that are now mostly infrequent, subjective, qualitative, memory-based, context-poor and sparse. Therefore, it is an ideal resource for physicians, physicians in training, software and hardware developers, computer scientists, data scientists, behavioural scientists, entrepreneurs, healthcare leaders and administrators who are seeking an up-to-date resource on this subject