Enrollment and Retention of Participants in Remote Digital Health Studies: Scoping Review and Framework Proposal

BACKGROUND Digital technologies are increasingly used in health research to collect real-world data from wider populations. A new wave of digital health studies relies primarily on digital technologies to conduct research entirely remotely. Remote digital health studies hold promise to significant cost and time advantages over traditional, in-person studies. However, such studies have been reported to typically suffer from participant attrition, the sources for which are still largely understudied. OBJECTIVE To contribute to future remote digital health study planning, we present a conceptual framework and hypotheses for study enrollment and completion. The framework introduces 3 participation criteria that impact remote digital health study outcomes: (1) participant motivation profile and incentives or nudges, (2) participant task complexity, and (3) scientific requirements. The goal of this study is to inform the planning and implementation of remote digital health studies from a person-centered perspective. METHODS We conducted a scoping review to collect information on participation in remote digital health studies, focusing on methodological aspects that impact participant enrollment and retention. Comprehensive searches were conducted on the PubMed, CINAHL, and Web of Science databases, and additional sources were included in our study from citation searching. We included digital health studies that were fully conducted remotely, included information on at least one of the framework criteria during recruitment, onboarding or retention phases of the studies, and included study enrollment or completion outcomes. Qualitative analyses were performed to synthesize the findings from the included studies. RESULTS We report qualitative findings from 37 included studies that reveal high values of achieved median participant enrollment based on target sample size calculations, 128% (IQR 100%-234%), and median study completion, 48% (IQR 35%-76%). Increased median study completion is observed for studies that provided incentives or nudges to extrinsically motivated participants (62%, IQR 43%-78%). Reducing task complexity for participants in the absence of incentives or nudges did not improve median study enrollment (103%, IQR 102%-370%) or completion (43%, IQR 22%-60%) in observational studies, in comparison to interventional studies that provided more incentives or nudges (median study completion rate of 55%, IQR 38%-79%). Furthermore, there were inconsistencies in measures of completion across the assessed remote digital health studies, where only around half of the studies with completion measures (14/27, 52%) were based on participant retention throughout the study period. CONCLUSIONS Few studies reported on participatory factors and study outcomes in a consistent manner, which may have limited the evidence base for our study. Our assessment may also have suffered from publication bias or unrepresentative study samples due to an observed preference for participants with digital literacy skills in digital health studies. Nevertheless, we find that future remote digital health study planning can benefit from targeting specific participant profiles, providing incentives and nudges, and reducing study complexity to improve study outcomes

Approval of artificial intelligence and machine learning-based medical devices in the USA and Europe (2015–20): a comparative analysis

There has been a surge of interest in artificial intelligence and machine learning (AI/ML)-based medical devices. However, it is poorly understood how and which AI/ML-based medical devices have been approved in the USA and Europe. We searched governmental and non-governmental databases to identify 222 devices approved in the USA and 240 devices in Europe. The number of approved AI/ML-based devices has increased substantially since 2015, with many being approved for use in radiology. However, few were qualified as high-risk devices. Of the 124 AI/ML-based devices commonly approved in the USA and Europe, 80 were first approved in Europe. One possible reason for approval in Europe before the USA might be the potentially relatively less rigorous evaluation of medical devices in Europe. The substantial number of approved devices highlight the need to ensure rigorous regulation of these devices. Currently, there is no specific regulatory pathway for AI/ML-based medical devices in the USA or Europe. We recommend more transparency on how devices are regulated and approved to enable and improve public trust, efficacy, safety, and quality of AI/ML-based medical devices. A comprehensive, publicly accessible database with device details for Conformité Européene (CE)-marked medical devices in Europe and US Food and Drug Administration approved devices is needed

Promoting participation in remote digital health studies: An expert interview study

BACKGROUND Remote digital health studies are on the rise and promise to reduce the operational inefficiencies of in-person research. However, they encounter specific challenges in maintaining participation (enrollment and retention) due to their exclusive reliance on technology across all study phases. OBJECTIVE The goal of this study was to collect experts' opinions on how to facilitate participation in remote digital health studies. METHOD We conducted 13 semi-structured interviews with principal investigators, researchers, and software developers who had recent experiences with remote digital health studies. Informed by the Unified Theory of Acceptance and Use of Technology (UTAUT) framework, we performed a thematic analysis and mapped various approaches to successful study participation. RESULTS Our analyses revealed four themes: (1) study planning to increase participation, where experts suggest that remote digital health studies should be planned based on adequate knowledge of what motivates, engages, and disengages a target population; (2) participant enrollment, highlighting that enrollment strategies should be selected carefully, attached to adequate support, and focused on inclusivity; (3) participant retention, with strategies that minimize the effort and complexity of study tasks and ensure that technology is adapted and responsive to participant needs, and (4) requirements for study planning focused on the development of relevant guidelines to foster participation in future studies. CONCLUSIONS Our findings highlight the significant requirements for seamless technology and researcher involvement in enabling high remote digital health study participation. Future studies can benefit from collected experiences and the development of guidelines to inform planning that balances participant and scientific requirements

Performance of the Swiss Digital Contact-Tracing App Over Various SARS-CoV-2 Pandemic Waves: Repeated Cross-sectional Analyses

Background: Digital proximity-tracing apps have been deployed in multiple countries to assist with SARS-CoV-2 pandemic mitigation efforts. However, it is unclear how their performance and effectiveness were affected by changing pandemic contexts and new viral variants of concern. Objective: The aim of this study is to bridge these knowledge gaps through a countrywide digital proximity-tracing app effectiveness assessment, as guided by the World Health Organization/European Center for Prevention and Disease Control (WHO/ECDC) indicator framework to evaluate the public health effectiveness of digital proximity-tracing solutions. Methods: We performed a descriptive analysis of the digital proximity-tracing app SwissCovid in Switzerland for 3 different periods where different SARS-CoV-2 variants of concern (ie, Alpha, Delta, and Omicron, respectively) were most prevalent. In our study, we refer to the indicator framework for the evaluation of public health effectiveness of digital proximity-tracing apps of the WHO/ECDC. We applied this framework to compare the performance and effectiveness indicators of the SwissCovid app. Results: Average daily registered SARS-CoV-2 case rates during our assessment period from January 25, 2021, to March 19, 2022, were 20 (Alpha), 54 (Delta), and 350 (Omicron) per 100,000 inhabitants. The percentages of overall entered authentication codes from positive tests into the SwissCovid app were 9.9% (20,273/204,741), 3.9% (14,372/365,846), and 4.6% (72,324/1,581,506) during the Alpha, Delta, and Omicron variant phases, respectively. Following receipt of an exposure notification from the SwissCovid app, 58% (37/64, Alpha), 44% (7/16, Delta), and 73% (27/37, Omicron) of app users sought testing or performed self-tests. Test positivity among these exposure-notified individuals was 19% (7/37) in the Alpha variant phase, 29% (2/7) in the Delta variant phase, and 41% (11/27) in the Omicron variant phase compared to 6.1% (228,103/3,755,205), 12% (413,685/3,443,364), and 41.7% (1,784,951/4,285,549) in the general population, respectively. In addition, 31% (20/64, Alpha), 19% (3/16, Delta), and 30% (11/37, Omicron) of exposure-notified app users reported receiving mandatory quarantine orders by manual contact tracing or through a recommendation by a health care professional. Conclusions: In constantly evolving pandemic contexts, the effectiveness of digital proximity-tracing apps in contributing to mitigating pandemic spread should be reviewed regularly and adapted based on changing requirements. The WHO/ECDC framework allowed us to assess relevant domains of digital proximity tracing in a holistic and systematic approach. Although the Swisscovid app mostly worked, as reasonably expected, our analysis revealed room for optimizations and further performance improvements. Future implementation of digital proximity-tracing apps should place more emphasis on social, psychological, and organizational aspects to reduce bottlenecks and facilitate their use in pandemic contexts. Keywords: COVID-19; SARS-CoV-2; SwissCovid app; Switzerland; contact-tracing app; digital contact tracing; digital proximity; digital tool; exposure notification; mobile app; public health; surveillance; variant of concern

Challenges and best practices for digital unstructured data enrichment in health research: A systematic narrative review

Digital data play an increasingly important role in advancing health research and care. However, most digital data in healthcare are in an unstructured and often not readily accessible format for research. Unstructured data are often found in a format that lacks standardization and needs significant preprocessing and feature extraction efforts. This poses challenges when combining such data with other data sources to enhance the existing knowledge base, which we refer to as digital unstructured data enrichment. Overcoming these methodological challenges requires significant resources and may limit the ability to fully leverage their potential for advancing health research and, ultimately, prevention, and patient care delivery. While prevalent challenges associated with unstructured data use in health research are widely reported across literature, a comprehensive interdisciplinary summary of such challenges and possible solutions to facilitate their use in combination with structured data sources is missing. In this study, we report findings from a systematic narrative review on the seven most prevalent challenge areas connected with the digital unstructured data enrichment in the fields of cardiology, neurology and mental health, along with possible solutions to address these challenges. Based on these findings, we developed a checklist that follows the standard data flow in health research studies. This checklist aims to provide initial systematic guidance to inform early planning and feasibility assessments for health research studies aiming combining unstructured data with existing data sources. Overall, the generality of reported unstructured data enrichment methods in the studies included in this review call for more systematic reporting of such methods to achieve greater reproducibility in future studies

Interplay of Digital Proximity App Use and SARS-CoV-2 Vaccine Uptake in Switzerland: Analysis of Two Population-Based Cohort Studies

Objectives: Our study aims to evaluate developments in vaccine uptake and digital proximity tracing app use in a localized context of the SARS-CoV-2 pandemic.Methods: We report findings from two population-based longitudinal cohorts in Switzerland from January to December 2021. Failure time analyses and Cox proportional hazards regression models were conducted to assess vaccine uptake and digital proximity tracing app (SwissCovid) uninstalling outcomes.Results: We observed a dichotomy of individuals who did not use the SwissCovid app and did not get vaccinated, and who used the SwissCovid app and got vaccinated during the study period. Increased vaccine uptake was observed with SwissCovid app use (aHR, 1.51; 95% CI: 1.40–1.62 [CI-DFU]; aHR, 1.79; 95% CI: 1.62–1.99 [CSM]) compared to SwissCovid app non-use. Decreased SwissCovid uninstallation risk was observed for participants who got vaccinated (aHR, 0.55; 95% CI: 0.38–0.81 [CI-DFU]; aHR, 0.45; 95% CI: 0.27–0.78 [CSM]) compared to participants who did not get vaccinated.Conclusion: In evolving epidemic contexts, these findings underscore the need for communication strategies as well as flexible digital proximity tracing app adjustments that accommodate different preventive measures and their anticipated interactions

Interplay of digital proximity app use and SARS-CoV-2 vaccine uptake in Switzerland : analysis of two population-based cohort studies

Objectives: Our study aims to evaluate developments in vaccine uptake and digital proximity tracing app use in a localized context of the SARS-CoV-2 pandemic. Methods: We report findings from two population-based longitudinal cohorts in Switzerland from January to December 2021. Failure time analyses and Cox proportional hazards regression models were conducted to assess vaccine uptake and digital proximity tracing app (SwissCovid) uninstalling outcomes. Results: We observed a dichotomy of individuals who did not use the SwissCovid app and did not get vaccinated, and who used the SwissCovid app and got vaccinated during the study period. Increased vaccine uptake was observed with SwissCovid app use (aHR, 1.51; 95% CI: 1.40–1.62 [CI-DFU]; aHR, 1.79; 95% CI: 1.62–1.99 [CSM]) compared to SwissCovid app non-use. Decreased SwissCovid uninstallation risk was observed for participants who got vaccinated (aHR, 0.55; 95% CI: 0.38–0.81 [CI-DFU]; aHR, 0.45; 95% CI: 0.27–0.78 [CSM]) compared to participants who did not get vaccinated. Conclusion: In evolving epidemic contexts, these findings underscore the need for communication strategies as well as flexible digital proximity tracing app adjustments that accommodate different preventive measures and their anticipated interactions

​The SwissCovid Digital Proximity Tracing App after one year: Were expectations fulfilled?

Digital proximity tracing has been promoted as a major technological innovation for its potential added benefits of greater speed, wider reach and better scalability compared with traditional manual contact tracing. First launched in Switzerland on 25 June 2020, the SwissCovid digital proximity tracing app has now been in use for more than one year. In light of this milestone, we raise the questions: What is currently known about the role of SwissCovid in mitigating the pandemic? Were the expectations fulfilled? In this review, we will summarise the current state of the literature from empirical studies on the adoption, performance and effectiveness of SwissCovid. The review consists of three sections. The first section summarizes findings from effectiveness studies, which suggest that SwissCovid exposure notifications contributed to preventive actions in 76% of exposure notification recipients and were associated with a faster quarantine time in some SwissCovid user groups. The second describes the public perception and current state of adoption of SwissCovid in Switzerland in light of prevalent misconceptions and overemphasised expectations. the third places the evidence on SwissCovid in an international context. Specifically, we compare key performance indicators of SwissCovid, which are of similar magnitude as for digital proximity tracing apps from other European countries. Using findings from Switzerland, we subsequently derive a preliminary measure of the population-level effectiveness of digital proximity tracing apps. We estimate that exposure notifications may have contributed to the notification and identification of 500 to 1000 SARS-CoV-2-positive app users per month. We explore why this effectiveness estimation is somewhat lower when compared with Germany or the United Kingdom. In light of the presented evidence, we conclude that digital proximity tracing works well in specific contexts, such as in mitigating non-household spread. However, future applications of digital proximity tracing should invest into stakeholder onboarding and increased process automatization – without deviating from the principles of voluntariness and user privacy

Using Venn Diagrams to Evaluate Digital Contact Tracing: Panel Survey Analysis

Background: Mitigation of the spread of infection relies on targeted approaches aimed at preventing nonhousehold interactions. Contact tracing in the form of digital proximity tracing apps has been widely adopted in multiple countries due to its perceived added benefits of tracing speed and breadth in comparison to traditional manual contact tracing. Assessments of user responses to exposure notifications through a guided approach can provide insights into the effect of digital proximity tracing app use on managing the spread of SARS-CoV-2. Objective: The aim of this study was to demonstrate the use of Venn diagrams to investigate the contributions of digital proximity tracing app exposure notifications and subsequent mitigative actions in curbing the spread of SARS-CoV-2 in Switzerland. Methods: We assessed data from 4 survey waves (December 2020 to March 2021) from a nationwide panel study (COVID-19 Social Monitor) of Swiss residents who were (1) nonusers of the SwissCovid app, (2) users of the SwissCovid app, or (3) users of the SwissCovid app who received exposure notifications. A Venn diagram approach was applied to describe the overlap or nonoverlap of these subpopulations and to assess digital proximity tracing app use and its associated key performance indicators, including actions taken to prevent SARS-CoV-2 transmission. Results: We included 12,525 assessments from 2403 participants, of whom 50.9% (1222/2403) reported not using the SwissCovid digital proximity tracing app, 49.1% (1181/2403) reported using the SwissCovid digital proximity tracing app and 2.5% (29/1181) of the digital proximity tracing app users reported having received an exposure notification. Most digital proximity tracing app users (75.9%, 22/29) revealed taking at least one recommended action after receiving an exposure notification, such as seeking SARS-CoV-2 testing (17/29, 58.6%) or calling a federal information hotline (7/29, 24.1%). An assessment of key indicators of mitigative actions through a Venn diagram approach reveals that 30% of digital proximity tracing app users (95% CI 11.9%-54.3%) also tested positive for SARS-CoV-2 after having received exposure notifications, which is more than 3 times that of digital proximity tracing app users who did not receive exposure notifications (8%, 95% CI 5%-11.9%). Conclusions: Responses in the form of mitigative actions taken by 3 out of 4 individuals who received exposure notifications reveal a possible contribution of digital proximity tracing apps in mitigating the spread of SARS-CoV-2. The application of a Venn diagram approach demonstrates its value as a foundation for researchers and health authorities to assess population-level digital proximity tracing app effectiveness by providing an intuitive approach for calculating key performance indicators. Keywords: COVID-19; SARS-CoV-2; Venn diagram approach; contact tracing; digital contact tracing; digital health; exposure notification; key performance indicators; mHealth; mobile apps; tracing apps

Using Venn Diagrams to Evaluate Digital Contact Tracing: Panel Survey Analysis.

BACKGROUND Mitigation of the spread of infection relies on targeted approaches aimed at preventing nonhousehold interactions. Contact tracing in the form of digital proximity tracing apps has been widely adopted in multiple countries due to its perceived added benefits of tracing speed and breadth in comparison to traditional manual contact tracing. Assessments of user responses to exposure notifications through a guided approach can provide insights into the effect of digital proximity tracing app use on managing the spread of SARS-CoV-2. OBJECTIVE The aim of this study was to demonstrate the use of Venn diagrams to investigate the contributions of digital proximity tracing app exposure notifications and subsequent mitigative actions in curbing the spread of SARS-CoV-2 in Switzerland. METHODS We assessed data from 4 survey waves (December 2020 to March 2021) from a nationwide panel study (COVID-19 Social Monitor) of Swiss residents who were (1) nonusers of the SwissCovid app, (2) users of the SwissCovid app, or (3) users of the SwissCovid app who received exposure notifications. A Venn diagram approach was applied to describe the overlap or nonoverlap of these subpopulations and to assess digital proximity tracing app use and its associated key performance indicators, including actions taken to prevent SARS-CoV-2 transmission. RESULTS We included 12,525 assessments from 2403 participants, of whom 50.9% (1222/2403) reported not using the SwissCovid digital proximity tracing app, 49.1% (1181/2403) reported using the SwissCovid digital proximity tracing app and 2.5% (29/1181) of the digital proximity tracing app users reported having received an exposure notification. Most digital proximity tracing app users (75.9%, 22/29) revealed taking at least one recommended action after receiving an exposure notification, such as seeking SARS-CoV-2 testing (17/29, 58.6%) or calling a federal information hotline (7/29, 24.1%). An assessment of key indicators of mitigative actions through a Venn diagram approach reveals that 30% of digital proximity tracing app users (95% CI 11.9%-54.3%) also tested positive for SARS-CoV-2 after having received exposure notifications, which is more than 3 times that of digital proximity tracing app users who did not receive exposure notifications (8%, 95% CI 5%-11.9%). CONCLUSIONS Responses in the form of mitigative actions taken by 3 out of 4 individuals who received exposure notifications reveal a possible contribution of digital proximity tracing apps in mitigating the spread of SARS-CoV-2. The application of a Venn diagram approach demonstrates its value as a foundation for researchers and health authorities to assess population-level digital proximity tracing app effectiveness by providing an intuitive approach for calculating key performance indicators