Evaluating the impact of physical activity apps and wearables: interdisciplinary review

Background: Although many smartphone apps and wearables have been designed to improve physical activity, their rapidly evolving nature and complexity present challenges for evaluating their impact. Traditional methodologies, such as randomized controlled trials (RCTs), can be slow. To keep pace with rapid technological development, evaluations of mobile health technologies must be efficient. Rapid alternative research designs have been proposed, and efficient in-app data collection methods, including in-device sensors and device-generated logs, are available. Along with effectiveness, it is important to measure engagement (ie, users’ interaction and usage behavior) and acceptability (ie, users’ subjective perceptions and experiences) to help explain how and why apps and wearables work. Objectives: This study aimed to (1) explore the extent to which evaluations of physical activity apps and wearables: employ rapid research designs; assess engagement, acceptability, as well as effectiveness; use efficient data collection methods; and (2) describe which dimensions of engagement and acceptability are assessed. Method: An interdisciplinary scoping review using 8 databases from health and computing sciences. Included studies measured physical activity, and evaluated physical activity apps or wearables that provided sensor-based feedback. Results were analyzed using descriptive numerical summaries, chi-square testing, and qualitative thematic analysis. Results: A total of 1829 abstracts were screened, and 858 articles read in full. Of 111 included studies, 61 (55.0%) were published between 2015 and 2017. Most (55.0%, 61/111) were RCTs, and only 2 studies (1.8%) used rapid research designs: 1 single-case design and 1 multiphase optimization strategy. Other research designs included 23 (22.5%) repeated measures designs, 11 (9.9%) nonrandomized group designs, 10 (9.0%) case studies, and 4 (3.6%) observational studies. Less than one-third of the studies (32.0%, 35/111) investigated effectiveness, engagement, and acceptability together. To measure physical activity, most studies (90.1%, 101/111) employed sensors (either in-device [67.6%, 75/111] or external [23.4%, 26/111]). RCTs were more likely to employ external sensors (accelerometers: P=.005). Studies that assessed engagement (52.3%, 58/111) mostly used device-generated logs (91%, 53/58) to measure the frequency, depth, and length of engagement. Studies that assessed acceptability (57.7%, 64/111) most often used questionnaires (64%, 42/64) and/or qualitative methods (53%, 34/64) to explore appreciation, perceived effectiveness and usefulness, satisfaction, intention to continue use, and social acceptability. Some studies (14.4%, 16/111) assessed dimensions more closely related to usability (ie, burden of sensor wear and use, interface complexity, and perceived technical performance). Conclusions: The rapid increase of research into the impact of physical activity apps and wearables means that evaluation guidelines are urgently needed to promote efficiency through the use of rapid research designs, in-device sensors and user-logs to assess effectiveness, engagement, and acceptability. Screening articles was time-consuming because reporting across health and computing sciences lacked standardization. Reporting guidelines are therefore needed to facilitate the synthesis of evidence across disciplines

Towards a Holistic Approach to Designing Theory-based Mobile Health Interventions

Increasing evidence has shown that theory-based health behavior change interventions are more effective than non-theory-based ones. However, only a few segments of relevant studies were theory-based, especially the studies conducted by non-psychology researchers. On the other hand, many mobile health interventions, even those based on the behavioral theories, may still fail in the absence of a user-centered design process. The gap between behavioral theories and user-centered design increases the difficulty of designing and implementing mobile health interventions. To bridge this gap, we propose a holistic approach to designing theory-based mobile health interventions built on the existing theories and frameworks of three categories: (1) behavioral theories (e.g., the Social Cognitive Theory, the Theory of Planned Behavior, and the Health Action Process Approach), (2) the technological models and frameworks (e.g., the Behavior Change Techniques, the Persuasive System Design and Behavior Change Support System, and the Just-in-Time Adaptive Interventions), and (3) the user-centered systematic approaches (e.g., the CeHRes Roadmap, the Wendel's Approach, and the IDEAS Model). This holistic approach provides researchers a lens to see the whole picture for developing mobile health interventions

Integrating Taxonomies into Theory-Based Digital Health Interventions for Behavior Change: A Holistic Framework

Digital health interventions have been emerging in the last decade. Due to their interdisciplinary nature, digital health interventions are guided and influenced by theories (e.g., behavioral theories, behavior change technologies, persuasive technology) from different research communities. However, digital health interventions are always coded using various taxonomies and reported in insufficient perspectives. The inconsistency and incomprehensiveness will bring difficulty for conducting systematic reviews and sharing contributions among communities. Based on existing related work, therefore, we propose a holistic framework that embeds behavioral theories, behavior change technique (BCT) taxonomy, and persuasive system design (PSD) principles. Including four development steps, two toolboxes, and one workflow, our framework aims to guide digital health intervention developers to design, evaluate, and report their work in a formative and comprehensive way

Effect of Values and Technology Use on Exercise: Implications for Personalized Behavior Change Interventions

Technology has recently been recruited in the war against the ongoing obesity crisis; however, the adoption of Health & Fitness applications for regular exercise is a struggle. In this study, we present a unique demographically representative dataset of 15k US residents that combines technology use logs with surveys on moral views, human values, and emotional contagion. Combining these data, we provide a holistic view of individuals to model their physical exercise behavior. First, we show which values determine the adoption of Health & Fitness mobile applications, finding that users who prioritize the value of purity and de-emphasize values of conformity, hedonism, and security are more likely to use such apps. Further, we achieve a weighted AUROC of .673 in predicting whether individual exercises, and we also show that the application usage data allows for substantially better classification performance (.608) compared to using basic demographics (.513) or internet browsing data (.546). We also find a strong link of exercise to respondent socioeconomic status, as well as the value of happiness. Using these insights, we propose actionable design guidelines for persuasive technologies targeting health behavior modification

Designing Culture-Tailored Persuasive Technology to Promote Physical Activity

Physical inactivity has been recognized as one of the leading risk factors that account for cardiovascular disease, type-2 diabetes, stroke, hypertension, etc., with the World Health Organization labeling it as the fourth leading risk factor for global mortality. Research has shown that persuasive technology (PT) can be leveraged as a motivational/supportive tool in tackling the physical-inactivity problem. In particular, research shows that persuasive health applications (PHAs) are more likely to be effective if they are theorydriven and tailored to the target audience. Yet, most existing PHAs on the market are neither theory-driven nor tailored to the target audience. Rather, their designers often employ a one-size- fits-all approach. This makes it difficult to know what design decisions are effective or ineffective among a given target audience. To bridge this gap, I proposed a framework, called the "EMVE-DeCK Framework," grounded in Bandura's Triad of Reciprocal Determinism, for designing, implementing and evaluating tailored PT interventions. Basically, the EMVE-DeCK Framework employs "Theory" and "Technology" to explain and change "Behavior." Moreover, research shows that culture can be leveraged as a personalization mechanism for tailoring PHAs to the target users to make them more effective. However, there is limited cross-cultural research|grounded in theory and empirical evidence|on the effectiveness of culture-based tailoring, especially comparative studies involving understudied populations in the PT research landscape. Hence, using the Hofstede's cultural framework (individualism vs. collectivism), Social Cognitive Theory, Technology Acceptance Model and the EMVE-DeCK Framework, I conducted a number of comparative studies to understand the culture-speci c determinants of physical-activity behavior and the acceptance of a proposed PHA. I used the ndings to inform the design, implementation and evaluation of two versions of a tness app called BEN'FIT - personal version (PV) and social version (SV) - aimed to motivate bodyweight exercise at home. In this dissertation, using the EMVE-DeCK Framework and Canada/United States (individualist culture) and Nigeria (collectivist culture) as a case study, I describe: (1) the cross-cultural user studies and empirical f indings that informed the PT intervention; (2) the design and implementation of the culture-tailored PHA; (3) the evaluation of the overall and culture-tailoring e ectiveness of the PHA in a eld setting. Finally, based on empirical evidence, I present a set of validated PT design guidelines in the eld for designing and tailoring PHAs to users in the individualist and collectivist cultures. This dissertation makes three major contributions to PT research in the Human-Computer-Interaction domain. Firstly, it demonstrates how theory and culture can be employed in the design and development of PT interventions to motivate behavior change. Secondly, it reveals and validates in the eld how the individualist and collectivist cultures fundamentally differ in their motivational mechanism of behavior change. Thirdly, it provides an in-the- field validates PT design guidelines for developing tailored PHAs for the two main types of culture. In the physical-activity domain, the dissertation is the rst to conduct a theory-driven, in-the-fi eld cross-cultural PT research that focuses on an understudied population from Africa (Nigeria) and compare its ndings with those of a widely studied population from North America (Canada/United States)

Designing Culture-Tailored Persuasive Technology to Promote Physical Activity

Physical inactivity has been recognized as one of the leading risk factors that account for cardiovascular disease, type-2 diabetes, stroke, hypertension, etc., with the World Health Organization labeling it as the fourth leading risk factor for global mortality. Research has shown that persuasive technology (PT) can be leveraged as a motivational/supportive tool in tackling the physical-inactivity problem. In particular, research shows that persuasive health applications (PHAs) are more likely to be effective if they are theory-driven and tailored to the target audience. Yet, most existing PHAs on the market are neither theory-driven nor tailored to the target audience. Rather, their designers often employ a one-size-fits-all approach. This makes it difficult to know what design decisions are effective or ineffective among a given target audience. To bridge this gap, I proposed a framework, called the “EMVE-DeCK Framework,” grounded in Bandura’s Triad of Reciprocal Determinism, for designing, implementing and evaluating tailored PT interventions. Basically, the EMVE-DeCK Framework employs “Theory” and “Technology” to explain and change “Behavior.” Moreover, research shows that culture can be leveraged as a personalization mechanism for tailoring PHAs to the target users to make them more effective. However, there is limited cross-cultural research—grounded in theory and empirical evidence—on the effectiveness of culture-based tailoring, especially comparative studies involving understudied populations in the PT research landscape. Hence, using the Hofstede’s cultural framework (individualism vs. collectivism), Social Cognitive Theory, Technology Acceptance Model and the EMVE-DeCK Framework, I conducted a number of comparative studies to understand the culture-specific determinants of physical-activity behavior and the acceptance of a proposed PHA. I used the findings to inform the design, implementation and evaluation of two versions of a fitness app called BEN’FIT—personal version (PV) and social version (SV)—aimed to motivate bodyweight exercise at home. In this dissertation, using the EMVE-DeCK Framework and Canada/United States (individualist culture) and Nigeria (collectivist culture) as a case study, I describe: (1) the cross-cultural user studies and empirical findings that informed the PT intervention; (2) the design and implementation of the culture-tailored PHA; and (3) the evaluation of the overall and culture-tailoring effectiveness of the PHA in a field setting. Finally, based on empirical evidence, I present a set of validated PT design guidelines in the field for designing and tailoring PHAs to users in the individualist and collectivist cultures. This dissertation makes three major contributions to PT research in the Human-Computer-Interaction domain. Firstly, it demonstrates how theory and culture can be employed in the design and development of PT interventions to motivate behavior change. Secondly, it reveals and validates in the field how the individualist and collectivist cultures fundamentally differ in their motivational mechanism of behavior change. Thirdly, it provides an in-the-field validated PT design guidelines for developing tailored PHAs for the two main types of culture. In the physical-activity domain, the dissertation is the first to conduct a theory-driven, in-the-field cross-cultural PT research that focuses on an understudied population from Africa (Nigeria) and compare its findings with those of a widely studied population from North America (Canada/United States)