Framework to assess the quality of mHealth apps: a mixed-method international case study protocol

Introduction: Healthcare professionals (HCPs) often recommend their patients to use a specific mHealth app as part of health promotion, disease prevention and patient self-management. There has been a significant growth in the number of HCPs downloading and using mobile health (mHealth) apps. Most mHealth apps that are available in app stores employ a ‘star rating’ system. This is based on user feedback on an app, but is highly subjective. Thus, the identification of quality mHealth apps which are deemed fit for purpose can be a difficult task for HCPs. Currently, there is no unified, validated standard guidelines for assessment of mHealth apps for patient safety, which can be used by HCPs. The Modified Enlight Suite (MES) is a quality assessment framework designed to provide a means for HCPs to evaluate mHealth apps before they are recommended to patients. MES was adapted from the original Enlight Suite for international use through a Delphi method, followed by preliminary validation process among a population predominantly consisting of medical students. This study aims to evaluate the applicability and validity of the MES, by HCPs, in low, middle and high income country settings. Methods and analysis: MES will be evaluated through a mixed-method study, consisting of qualitative (focus group) and quantitative (survey instruments) research, in three target countries: Malaŵi (low income), South Africa (middle income) and Ireland (high income). The focus groups will be conducted through Microsoft Teams (Microsoft, Redmond, Washington, USA) and surveys will be conducted online using Qualtrics (Qualtrics International, Seattle, Washington, USA). Participants will be recruited through the help of national representatives in Malawi (Mzuzu University), South Africa (University of Fort Hare) and Ireland (University College Cork) by email invitation. Data analysis for the focus group will be by the means of thematic analysis. Data analysis for the survey will use descriptive statistics and use Cronbach alpha as an indicator of internal consistency of the MES. The construct validity of the mHealth app will be assessed by computing the confirmatory factor analysis using Amos. Ethics and dissemination: The study has received ethical approval from the Social Research Ethics Committee (SREC) SREC/SOM/03092021/1 at University College Cork, Ireland, Malaŵi Research Ethics Committee (MREC), Malaŵi MZUNIREC/DOR/21/59 and Inter-Faculty Research Ethics Committee (IFREC) of University of Fort Hare (REC-2 70 710-028-RA). The results of the study will be disseminated through the internet, peer-reviewed journals and conference presentations

The Importance of form field validation: lessons learnt from a feasibility study of an mHealth application in Malawi, Africa

Measuring adherence to clinical guidelines using mobile health (mHealth) technologies when form field validation is enforced or turned on could potentially be viewed as skewing the dataset, leading to 100% adherence to the clinical rule base. In theory, healthcare providers should fully abide by clinical guidelines, whether in paper or digital format, to ensure that the patient receives appropriate care. However, what happens when mHealth form field validation is turned off? As part of a feasibility study in Malawi, Africa, we explored this phenomenon. Switching off validation on the mHealth artefact served its purpose within the context of a feasibility study where a parallel paper-based clinical assessment process remained in place. The design of this technical artefact with the turnkey validation feature afforded us the opportunity to turn validation on and off seamlessly. Ultimately, from an ethical, clinical and technical perspective the optimum approach is to ensure that form field validation is switched on. With form field validation on adherence to the clinical guidelines is enforced which minimises incomplete assessment and the potential for suboptimal clinical decisions that could adversely affect patient care

Effectiveness of smartphone-based community case management on the urgent referral, reconsultation, and hospitalization of children aged under 5 years in Malawi: cluster-randomized, stepped-wedge trial

Background: Integrated community case management (CCM) has led to reductions in child mortality in Malawi from illnesses such as malaria, pneumonia and diarrhoea. However, adherence to CCM guidelines is often poor, potentially leading to inappropriate clinical decisions and poor outcomes. We determined the impact of an electronic version of a smartphone-based CCM (eCCM) application on referral, re-consultation and hospitalization rates of children presenting to village clinics in Malawi. Methods: A stepped-wedge cluster-randomized trial compared paper-based CCM (control) with and without use of an eCCM app on smartphones from November 2016 to February 2017. A total of 102 village clinics from two districts in Northern Malawi were assigned to one of six clusters which were randomized to the sequencing of crossover from the control to the intervention phases, as well as the duration of exposure in each phase. Children ≥2 months to 0.05). Conclusions: Addition of eCCM decision support using smartphones led to a greater proportion of children being referred to higher-level facilities, with no apparent increase in hospital admissions or repeat consultations in village clinics. Our findings provide support for the implementation of eCCM tools in Malawi and other Low and Middle Income Countries (LMIC), with a need for ongoing assessment of effectiveness and integration with national digital health strategies. Trial registration: ClinicalTrials.gov; NCT02763345. Registered 3 May 201

End-user perspectives of two mHealth decision support tools: Electronic Community Case Management in Northern Malawi

Background The introduction of a paper-based Community Case Management (CCM) in Malawi has contributed to a reduction of child morbidity and mortality rates. In addition, the introduction of electronic Community Case Management (eCCM) (smartphones with built in CCM apps) may help to reduce the under-five mortality rates even further. Purpose It is not uncommon for Apps with a similar area of interest to develop different features to assist the end users. Such differences between Apps may have a significant role to play in its overall adoption and integration. The purpose of this research was to explore end users perspectives of two eCCM decision support tools developed and implemented by the Supporting LIFE project (SL eCCM App) and D-Tree International’s (Mangologic eCCM App)in Northern Malawi. Methods A mixed methods approach was applied, involving a survey of 109 users (106 Health Surveillance Assistants (HSAs), and 3 Integrated Management of Childhood Il6lnesses (IMCI) coordinators). This was followed up with semi-structured interviews with 34 respondents (31 HSAs, and 3 IMCI coordinators). Quantitative data was analyzed using SPSS version 20 where descriptive statistics and Chi-Squared tests were generated. Qualitative data were analyzed based on thematic analysis. Results Participants reported that both Apps could assist the HSAs in the management of childhood illnesses. However, usability differed between the two apps where the Supporting LIFE eCCM App was found to be easier to use (61%) compared to the Mangologic eCCM App (4%). Both Apps were perceived to provide credible and accurate information. Conclusion It is essential that the quality of the data within Mobile Health (mHealth) Apps is high, however even Apps with excellent levels of data quality may not succeed if the overall usability of the App is low. Therefore it is essential that the Apps has high levels of data quality, usability and credibility. The results of this study will help inform mobile Health (mHealth) App designers in developing future eCCM Apps as well as researchers and policy makers when considering the adoption of mHealth solutions in the future in Malawi and other LMICs

The added value of a mobile application of Community Case Management on under-5 referral, re-consultation and hospitalization rates in two districts in Northern Malawi: the Supporting LIFE study protocol for a pragmatic cluster-randomized controlled trial

Background: There is evidence to suggest frontline community health workers in Malawi are under-referring children with acute illness to higher-level facilities. Integrating a digitized version of paper-based methods of delivering Community Case Management (CCM) could strengthen delivery, increasing urgent referral rates and preventing unnecessary re-consultations and hospital admissions. This trial aims to evaluate the added value of the Supporting LIFE electronic Community Case Management Application (SL eCCM App) compared to paper-based CCM on urgent referral, re-consultation and hospital admission rates, in two districts in Northern Malawi. Methods: This is a pragmatic stepped-wedge cluster randomized trial assessing the added value of the SL eCCM App on urgent referral, re-consultation, and hospitalization rates of children aged ≥2 months to <5 years, within 7-days of the index visit. 102 HSAs were stratified into six clusters based on geographical location, and clusters randomized to the timing of crossover to the intervention using simple computer-generated randomization. Training workshops were conducted prior to the control (paper-CCM) and intervention (paper-CCM + SL eCCM App) in assigned clusters. Neither participants nor study personnel were blinded to allocation. Outcome measures were determined by abstraction of clinical data from patient records 2 weeks after recruitment. A nested qualitative study explored perceptions of adherence to urgent referral recommendations, and a cost evaluation determined the financial and time-related costs to caregivers of subsequent healthcare utilization. The trial was conducted between July 2016 and February 2017. Discussion: This is the first large-scale trial evaluating the value of adding a mobile application of CCM to the assessment of acutely ill children under-5. The trial will generate evidence on the potential use of mobile health for CCM in Malawi, and more widely in other developing countries