Evaluating the impact of physical activity apps and wearables: interdisciplinary review

Background: Although many smartphone apps and wearables have been designed to improve physical activity, their rapidly evolving nature and complexity present challenges for evaluating their impact. Traditional methodologies, such as randomized controlled trials (RCTs), can be slow. To keep pace with rapid technological development, evaluations of mobile health technologies must be efficient. Rapid alternative research designs have been proposed, and efficient in-app data collection methods, including in-device sensors and device-generated logs, are available. Along with effectiveness, it is important to measure engagement (ie, users’ interaction and usage behavior) and acceptability (ie, users’ subjective perceptions and experiences) to help explain how and why apps and wearables work. Objectives: This study aimed to (1) explore the extent to which evaluations of physical activity apps and wearables: employ rapid research designs; assess engagement, acceptability, as well as effectiveness; use efficient data collection methods; and (2) describe which dimensions of engagement and acceptability are assessed. Method: An interdisciplinary scoping review using 8 databases from health and computing sciences. Included studies measured physical activity, and evaluated physical activity apps or wearables that provided sensor-based feedback. Results were analyzed using descriptive numerical summaries, chi-square testing, and qualitative thematic analysis. Results: A total of 1829 abstracts were screened, and 858 articles read in full. Of 111 included studies, 61 (55.0%) were published between 2015 and 2017. Most (55.0%, 61/111) were RCTs, and only 2 studies (1.8%) used rapid research designs: 1 single-case design and 1 multiphase optimization strategy. Other research designs included 23 (22.5%) repeated measures designs, 11 (9.9%) nonrandomized group designs, 10 (9.0%) case studies, and 4 (3.6%) observational studies. Less than one-third of the studies (32.0%, 35/111) investigated effectiveness, engagement, and acceptability together. To measure physical activity, most studies (90.1%, 101/111) employed sensors (either in-device [67.6%, 75/111] or external [23.4%, 26/111]). RCTs were more likely to employ external sensors (accelerometers: P=.005). Studies that assessed engagement (52.3%, 58/111) mostly used device-generated logs (91%, 53/58) to measure the frequency, depth, and length of engagement. Studies that assessed acceptability (57.7%, 64/111) most often used questionnaires (64%, 42/64) and/or qualitative methods (53%, 34/64) to explore appreciation, perceived effectiveness and usefulness, satisfaction, intention to continue use, and social acceptability. Some studies (14.4%, 16/111) assessed dimensions more closely related to usability (ie, burden of sensor wear and use, interface complexity, and perceived technical performance). Conclusions: The rapid increase of research into the impact of physical activity apps and wearables means that evaluation guidelines are urgently needed to promote efficiency through the use of rapid research designs, in-device sensors and user-logs to assess effectiveness, engagement, and acceptability. Screening articles was time-consuming because reporting across health and computing sciences lacked standardization. Reporting guidelines are therefore needed to facilitate the synthesis of evidence across disciplines

Varhaisnuorille suunnatun aktivoivan digitaalisen peli-intervention soveltuvuustutkimus

Lapsen terve kasvu ja kehitys vaatii riittävää fyysistä aktiivisuutta, ja fyysisellä aktiivisuudella voidaan ehkäistä monia terveysriskejä. Liikunnallinen elämäntapa lapsuudessa jatkuu usein myös aikuisena. Fyysisen aktiivisuuden määrä vähenee usein varhaisnuoruudessa murrosiän kynnyksellä, minkä vuoksi varhaisnuorille suunnatut fyysistä aktiivisuutta lisäävät interventiot ovat tarpeellisia. Tämän tutkimuksen tarkoituksena oli arvioida varhaisnuorten fyysisen aktiivisuuden edistämiseksi kehitetyn aktivoivan digitaalisen peli-intervention soveltuvuutta. Soveltuvuustutkimuksen avulla arvioitiin intervention käytettävyyttä, hyväksyttävyyttä, alustavaa vaikuttavuutta ja tutkimuksen toteutettavuutta. Tutkimuksen kohderyhmänä olivat perusopetuksen 10–13-vuotiaat varhaisnuoret. Tutkimukseen osallistui yhteensä 38 tutkittavaa, jotka satunnaistettiin interventio- ja kontrolliryhmään. Interventioryhmässä oli 14 tutkittavaa ja kontrolliryhmässä 33 tutkittavaa. Pelin toimintaongelmien vuoksi dataa kertyi interventioryhmässä vain kuudelta tutkittavalta. Tutkittavien fyysistä aktiivisuutta mitattiin Actigraph -kiihtyvyysmittareilla ja fyysisen aktiivisuuden minäpystyvyyttä PASES-kyselyllä, ennen ja jälkeen intervention. Lisäksi tutkittavat vastasivat kyselyyn sovelluksen käytettävyydestä ja hyväksyttävyydestä. Tutkimuksessa kerättiin myös tietoa tutkittavien pelisovelluksen käytöstä ja koko tutkimusprosessin onnistumisesta. Varhaisnuoret arvioivat pelin käytettävyyden SUS-testillä mediaaniin 60p, joka jäi alle SUS-testin käytettävyyden tavoitearvon (68p.). Pelin hyväksyttävyyden pelaajat kokivat yleisesti ottaen positiivisena. Fyysisen aktiivisuuden määrä väheni sekä interventio- että kontrolliryhmällä loppumittauksissa alkumittauksiin verrattuna, mutta kiihtyvyysmittareiden käyttö oli loppumittauksissa heikkoa, mikä vaikutti tulosten luotettavuuteen. PASES-minäpystyvyyskyselyn tulokset paranivat interventioryhmällä hieman, mutta eivät tilastollisesti merkitsevästi. Tutkimuksen toteutettavuutta heikensi pelisovelluksen toimintaongelmat. Lisäksi rekrytointi oli haasteellista ja tutkittavien kiihtyvyysmittareiden käyttö erityisesti loppumittauksissa jäi alle suositusten. Nämä tekijät heikensivät myös tutkimustulosten luotettavuutta. Tutkimuksen perusteella saatiin tietoa intervention soveltuvuudesta. Tulosten perusteella tutkimuksen laajempi toteuttaminen on mahdollista, mutta toteutuksessa tulee huomioida tämän tutkimuksen tulokset. Intervention vaikuttavuuden arvioimiseksi vaaditaan laajemmalla otoksella toteutettu tutkimus. Tutkimukset varhaisnuorten fyysisen aktiivisuuden lisäämisen keinoista ovat tarpeen.Adequate and sufficient physical activity is an essential part of healthy growth and development of children and it can also prevent many health risks. A healthy and active lifestyle learnt during childhood often continues into adulthood. Children’s physical activity declines particularly on the threshold of puberty, which is why interventions for increasing physical activity of the preadolescents is required. The purpose of this study was to evaluate the feasibility of the Movenator excergame intervention for promoting physical activity of the preadolescents. The feasibility study was used to assess the usability, acceptability and preliminary effectiveness of the intervention and the feasibility of the research process. The target group of the study were 10–13-year-old preadolescent school children. 38 participants were recruited, and they were randomized to intervention and control group. There were 14 participants in the intervention group and 33 participants in the control group. Due to technical problems, data was only gathered for six participants in the intervention group. The physical activity of the participants was measured by Actigraph -accelerometers. The physical activity self-efficacy was measured by PASES -questionnaire. The participants also answered questionnaires about the usability and acceptability of the game application. Data was also gathered for the use of the game application and the overall success of the research process. The usability of the game application was rated by System Usability Scale (SUS) for a median score of 60 points. The score is below 68 points, which is the targeted score of the SUS-test. The participants evaluated the acceptability of the game application positively in general. The physical activity of the participants declined in both the intervention and the control groups in the follow-up accelerometry compared to baseline. However, the compliance with the accelerometer at the follow-up was poor which can affect to the reliability of the results. There was a small positive intervention effect to physical activity self-efficacy in the intervention group, but the effect was not statistically significant. The feasibility of this study was reduced by the technical problems of the game application. The recruiting process was demanding and the compliance with the accelerometer usage was below the recommendations at the follow up. These factors also decreased the reliability of the results. This study provided information about the feasibility and preliminary efficiency of the Movenator -excergame intervention. The results indicated that a larger scale intervention study is conceivable, however the results of this feasibility study must be considered. A larger sample size is needed to evaluate the intervention effect. More research is necessary to discern how to promote the physical activity in preadolescence

A field experiment on gamification of physical activity – effects on motivation and steps

Gamification is finding growing application in the field of physical activity, promising engaging and motivating experiences that foster behavioural change. Yet, rigorous empirical work substantiating favourable claims is limited. Our study sought to find evidence for the effects resulting from gamification of physical activity on the users’ motivation, perceived usefulness, and the behavioural change in terms of step counts. We employed a four-week randomised controlled field experiment to investigate how the choice of different gamification designs affects outcomes. Participants were provided with a wearable physical activity tracker and randomly assigned to either a: 1) competitive gamified group; 2) cooperative gamified group; 3) hybrid gamified group; or 4) control group. Data gathered includes a panel dataset of step counts and self-reported data on the users’ intrinsic motivation and perceived usefulness. We found that at the end of the intervention, gamification made no difference to self-assessed intrinsic motivation or perceived usefulness compared to a non-gamified self-tracking experience. Yet, despite the lack of psychological effects, the use of gamification did result in stronger behavioural outcomes relative to the control group - in the form of increased step counts. Indeed, all groups treated with gamification recorded an increase in step counts during the intervention period. Furthermore, among the gamified treatments, it was the hybrid design that generated the largest difference in step counts (relative to the control group). The finding that gamification can stimulate a stronger behavioural outcome without stimulating a stronger psychological one merits further investigation as to the mechanisms at play.peer-reviewe

Association of exergaming with physical literacy in Canadian children

Prolonged sedentary time in children and youth is associated with decreased physical fitness, motor competence and overall health. Active video games (e.g. exergames) have been suggested as a more active alternative to sedentary screen time. While there is research examining the effect of video game use on specific physiological outcomes, there is limited evidence exploring how they affect the development of physical literacy. This study examined physical literacy using Canadian Assessment of Physical Literacy (CAPL) self-report questionnaires from 317 eight to thirteen year old Lethbridge area children. Independent sample t-tests were used to determine differences between genders as well as exergame users and non-users. Multiple linear regression analyses were conducted to determine if a relationship was present between exergaming time and the overall physical literacy score as well as the individual domains (physical competence, knowledge and understanding, motivation and confidence, and daily behavior). A discriminate function analysis was used to determine if CAPL interpretation group membership could be predicted by weekly exergame use. Sedentary screen-based activities were inversely related to the overall physical literacy score (P>.001), whereas exergaming time was not significantly associated with overall physical literacy. Self-reported exergaming time was negatively associated with knowledge and understanding and positively related to motivation and confidence. Our results suggest that increased screen time, regardless of exergame use, is detrimental to overall physical literacy. However, exergaming has the potential to increase children’s motivation and confidence.MITACS University of Lethrbidg

Digitales Lernspiel als Hilfsmittel zur Überwindung von Barrieren zur Früherkennung und Behandlung kindlicher Adipositas – Ergebnisse eines systematischen Reviews und Evaluation von Modul 1 in einer Pilotstudie

In Zeiten der global zunehmenden Adipositas in der Bevölkerung steigt auch die Zahl der übergewichtigen Kinder an. Übergewicht im Kindesalter kann zu schwerwiegenden gesundheitlichen Beeinträchtigungen, insbesondere im späteren Erwachsenenleben, führen. Um diesem Trend entgegenzuwirken, werden in dieser Dissertation neue Therapie- und Präventionsmöglichkeiten aus dem Bereich der digitalen Medien untersucht. In Form einer Pilotstudie wurde das Modul 1 des Lernspiels „kids obestiy prevention, KOP“ getestet. Das Konzept für KOP wurde von der Abteilung für Psychosomatische Medizin und Psychotherapie der Universitätsklinik Tübingen und des Cluster 7 des WissenschaftsCampus Tübingen entwickelt. Das Ziel des Konzeptes ist es, mit Hilfe von modernen Medien, einen Zugang zu Kindern der Altersgruppe 7-15 Jahren zu finden, um kindlichem Übergewicht vorzubeugen und die Therapie desselben zu unterstützen. Diese Pilotstudie stellt die Grundlage für eine zukünftige randomisiert kontrollierte Hauptstudie mit größerer Stichprobe dar. Die Praktikabilität und Akzeptanz, das Verständnis, ein möglicher Wissenserwerb und der Spielspaß von KOP standen in dieser Untersuchung im Vordergrund. Die Pilotstudie wurde in einer dritten Klasse in einer Tübinger Grundschule durchgeführt. Es nahmen 17 Schulkinder im Alter von acht bis zehn Jahren teil (♀ n=9, ♂ n=8). Im zeitlichen Rahmen von circa drei Wochen, am Ende des Schuljahres 2013/2014, wurden mit jedem teilnehmenden Kind ein Wissenstest vor und nach dem Spiel, sowie ein abschließendes Interview durchgeführt und das erste Modul des Lernspiels KOP durchgespielt. Das Spiel konnte ohne nennenswerte technische Komplikationen durchgeführt werden. Es wurde bereits in dieser frühen Entwicklungsphase von den Kindern gut akzeptiert, zeigt einen signifikanten positiven Effekt auf den Wissenserwerb und bietet dem spielenden Kind ein moderates Maß an Bewegung. Im Rahmen dieser Pilotstudie wurden Verbesserungsmöglichkeiten der Spielprogrammierung bzw. des Studienablaufs eruiert. Mit wenigen Ausnahmen u.a. des Begriffs „Energiedichte“ waren alle Spielinhalte und Fragen im Wissenstest den Kindern gut verständlich. Die Ergebnisse der Pilotstudie weisen darauf hin, dass der Einsatz von digitalen Lernspielen in der Präventions- und Therapie von kindlicher Adipositas als eine ergänzende Maßnahme sinnvoll sein können. Sie bieten eine kindgerechte Möglichkeit, Übergewicht bei Kindern mit Hilfe moderner Medien lösungsorientiert anzugehen. Ergänzend zur empirischen Untersuchung wurde eine Literaturübersicht zu diesem Thema in Form eines systematischen Review mitverfasst: „Chances and limitations of video games in the fight of childhood obesity – a systematic review”. Dafür wurden Studien mit Video- oder Computerspielen gesucht, die sich mit den Themen Ernährung, Bewegung und kindlichem Übergewicht auseinander setzen. Die Zielgruppe waren Kinder im Alter zwischen 5 und 15 Jahren. Aus 2379 Quellen wurden 64 Studien zu diesem Thema ausgewählt und ausgewertet. Die meisten dieser Studien konnten qualitativ positive Effekte in ihren Ergebnissen zeigen. Diese zeigten sich u.a. in einer Verbesserung von gewichtsrelevanten Kennwerten, körperlicher Aktivität/Fitness, Ernährungsgewohnheiten oder in einem Zuwachs von Wissen in diesen Bereichen. Allerdings waren die gefunden Effekte nur klein. Psychosoziale Aspekte wurden in den Studien nicht berücksichtigt. Die Eltern wurden nur in einem kleinen Teil der Studien aktiv integriert. Es lässt sich aus den Ergebnissen schlussfolgern, dass der alleinige Einsatz von Video- oder Computerspielen zur Gewichtsreduktion bzw. –stabilisation keine zufriedenstellenden Ergebnisse liefert. Allerdings könnte ihr Einsatz als ergänzende Komponente eine vielversprechende Möglichkeit sein, um Präventions- und Therapieprogramme der kindlichen Adipositas erfolgreicher zu machen. Sie könnten einen positiven Einfluss auf die Akzeptanz der einzelnen Programme nehmen und dadurch die Ergebnisse bzw. Therapieerfolge verbessern

Evaluating the impact of physical activity apps and wearables: an interdisciplinary investigation of research designs and methods

Many smartphone apps and wearables have been developed to promote physical activity, however there are challenges in assessing their impact. Apps and wearables are rapidly evolving technologies and thousands of physical activity apps that are publicly available on app stores remain unevaluated. There are concerns that traditional “gold standard” evaluation approaches, such as randomised control trials (RCTs), may be too slow to keep up with these, and produce effectiveness results that do not reflect real world settings. Rapid research designs (such as single case designs; SCDs) and innovative data collection methods (in-device sensors, device-generated user logs) have been proposed to improve research efficiency, yet preliminary evidence suggests they are not widely used in mHealth. This thesis reports three studies undertaken to investigate the use of rapid research designs and efficient methods for evaluating physical activity apps and wearables. First, a scoping review of the extent to which these approaches are employed by health and HCI researchers. Second, semi-structured interviews with researchers, data scientists and industry professionals to provide a deeper understanding of current evaluation practices. Third, the development and refinement of a methodological framework to support researchers in using SCDs in automated app store evaluations of physical activity apps. The findings suggest rapid research designs are not often employed in evaluations of physical activity and other health behaviour change apps. Researchers feel they face opportunity barriers (e.g. risking not being funded or published) and do not have the necessary skills (e.g. in using device generated user logs). Industry professionals appear to lack the motivation and time to evaluate effectiveness. Trade-offs were perceived between the measurement accuracy of in-device sensors and other factors such as user burden. Automated trials may speed up evaluations of physical activity apps and wearables, and the suggested data collection framework aims to support researchers in conducting rigorous effectiveness evaluations using app store- based SCDs. However, further work is needed to enable industry professionals to use the framework to evaluate their publicly-available apps