The use of mediation analysis in evaluation of complex health interventions

This article presents an application of the causal inference approach to mediation analysis using the example of a complex intervention that aimed to improve the quality of care at health centres in Uganda. Mediation analysis is a statistical method that aims to isolate the causal mechanisms that make an intervention work in a given context. We combined data from a cluster randomized control trial and a mixed-methods process evaluation. We developed two causal models following our hypotheses of how the intervention was intended to work through mechanisms at health centres to improve health outcomes in the community. In adjusted analyses, there was evidence of an effect of the intervention on some health centre mechanisms; however, these did not lead to improvements in community health outcomes. We discuss the practical and epistemological challenges encountered when using mediation analysis to evaluate a complex intervention. These findings will inform future evaluations. Trial registration: The trial reported in this article is registered at: clinicaltrials.gov, NCT01024426. Registered 2 December 2009, https://clinicaltrials.gov/ct2/show/record/NCT01024426?term=NCT01024426&draw=2&rank=

Exploring Barriers and Facilitators of Adherence to Artemisinin-Based Combination Therapies for the Treatment of Uncomplicated Malaria in Children in Freetown, Sierra Leone.

Medication adherence is an essential step in the malaria treatment cascade. We conducted a qualitative study embedded within a randomized controlled trial comparing the adherence to the recommended dosing of two artemisinin-based combination therapies (ACT) to treat uncomplicated malaria in Freetown, Sierra Leone. This study explored the circumstances and factors that influenced caregiver adherence to the ACT prescribed for their child in the trial. In-depth interviews were conducted with 49 caregivers; all interviews were recorded, transcribed, and translated. Transcripts were coded and aggregated into themes, applying a thematic content approach. We identified four key factors that influenced optimal treatment adherence: (1) health system influences, (2) health services, (3) caregivers' experiences with malaria illness and treatment, and (4) medication characteristics. Specifically, caregivers reported confidence in the health system as facilities were well maintained and care was free. They also felt that health workers provided quality care, leading them to trust the health workers and believe the test results. Ease of medication administration and perceived risk of side effects coupled with caregivers' prior experience treating malaria influenced how medications were administered. To ensure ACTs achieve maximum effectiveness, consideration of these contextual factors and further development of child-friendly antimalarials are needed

The impact of an intervention to introduce malaria rapid diagnostic tests on fever case management in a high transmission setting in Uganda: A mixed-methods cluster-randomized trial (PRIME).

Rapid diagnostic tests for malaria (mRDTs) have been scaled-up widely across Africa. The PRIME study evaluated an intervention aiming to improve fever case management using mRDTs at public health centers in Uganda. A cluster-randomized trial was conducted from 2010-13 in Tororo, a high malaria transmission setting. Twenty public health centers were randomized in a 1:1 ratio to intervention or control. The intervention included training in health center management, fever case management with mRDTs, and patient-centered services; plus provision of mRDTs and artemether-lumefantrine (AL) when stocks ran low. Three rounds of Interviews were conducted with caregivers of children under five years of age as they exited health centers (N = 1400); reference mRDTs were done in children with fever (N = 1336). Health worker perspectives on mRDTs were elicited through semi-structured questionnaires (N = 49) and in-depth interviews (N = 10). The primary outcome was inappropriate treatment of malaria, defined as the proportion of febrile children who were not treated according to guidelines based on the reference mRDT. There was no difference in inappropriate treatment of malaria between the intervention and control arms (24.0% versus 29.7%, adjusted risk ratio 0.81 95\% CI: 0.56, 1.17 p = 0.24). Most children (76.0\%) tested positive by reference mRDT, but many were not prescribed AL (22.5\% intervention versus 25.9\% control, p = 0.53). Inappropriate treatment of children testing negative by reference mRDT with AL was also common (31.3\% invention vs 42.4\% control, p = 0.29). Health workers appreciated mRDTs but felt that integrating testing into practice was challenging given constraints on time and infrastructure. The PRIME intervention did not have the desired impact on inappropriate treatment of malaria for children under five. In this high transmission setting, use of mRDTs did not lead to the reductions in antimalarial prescribing seen elsewhere. Broader investment in health systems, including infrastructure and staffing, will be required to improve fever case management

The Impact of an Intervention to Improve Malaria Care in Public Health Centers on Health Indicators of Children in Tororo, Uganda (PRIME): A Cluster-Randomized Trial.

Optimizing quality of care for malaria and other febrile illnesses is a complex challenge of major public health importance. To evaluate the impact of an intervention aiming to improve malaria case management on the health of community children, a cluster-randomized trial was conducted from 2010-2013 in Tororo, Uganda, where malaria transmission is high. Twenty public health centers were included; 10 were randomized in a 1:1 ratio to intervention or control. Households within 2 km of health centers provided the sampling frame for the evaluation. The PRIME intervention included training in fever case management using malaria rapid diagnostic tests (mRDTs), patient-centered services, and health center management; plus provision of mRDTs and artemether-lumefantrine. Cross-sectional community surveys were conducted at baseline and endline (N = 8,766), and a cohort of children was followed for approximately 18 months (N = 992). The primary outcome was prevalence of anemia (hemoglobin < 11.0 g/dL) in children under 5 years of age in the final community survey. The intervention was delivered successfully; however, no differences in prevalence of anemia or parasitemia were observed between the study arms in the final community survey or the cohort. In the final survey, prevalence of anemia in children under 5 years of age was 62.5% in the intervention versus 63.1% in control (adjusted risk ratio = 1.01; 95% confidence interval = 0.91-1.13; P = 0.82). The PRIME intervention, focusing on training and commodities, did not produce the expected health benefits in community children in Tororo. This challenges common assumptions that improving quality of care and access to malaria diagnostics will yield health gains

Strengthening patient-centred communication in rural Ugandan health centres: A theory-driven evaluation within a cluster randomized trial.

This article describes a theory-driven evaluation of one component of an intervention to improve the quality of health care at Ugandan public health centres. Patient-centred services have been advocated widely, but such approaches have received little attention in Africa. A cluster randomized trial is evaluating population-level outcomes of an intervention with multiple components, including 'patient-centred services.' A process evaluation was designed within this trial to articulate and evaluate the implementation and programme theories of the intervention. This article evaluates one hypothesized mechanism of change within the programme theory: the impact of the Patient Centred Services component on health-worker communication. The theory-driven approach extended to evaluation of the outcome measures. The study found that the proximal outcome of patient-centred communication was rated 10 percent higher (p < 0.008) by care seekers consulting with the health workers who were at the intervention health centres compared with those at control health centres. This finding will strengthen interpretation of more distal trial outcomes

The practice of 'doing' evaluation: Lessons learned from nine complex intervention trials in action

Background: There is increasing recognition among trialists of the challenges in understanding how particular 'real-life' contexts influence the delivery and receipt of complex health interventions. Evaluations of interventions to change health worker and/or patient behaviours in health service settings exemplify these challenges. When interpreting evaluation data, deviation from intended intervention implementation is accounted for through process evaluations of fidelity, reach, and intensity. However, no such systematic approach has been proposed to account for the way evaluation activities may deviate in practice from assumptions made when data are interpreted.Methods: A collective case study was conducted to explore experiences of undertaking evaluation activities in the real-life contexts of nine complex intervention trials seeking to improve appropriate diagnosis and treatment of malaria in varied health service settings. Multiple sources of data were used, including in-depth interviews with investigators, participant-observation of studies, and rounds of discussion and reflection.Results and discussion: From our experiences of the realities of conducting these evaluations, we identified six key 'lessons learned' about ways to become aware of and manage aspects of the fabric of trials involving the interface of researchers, fieldworkers, participants and data collection tools that may affect the intended production of data and interpretation of findings. These lessons included: foster a shared understanding across the study team of how individual practices contribute to the study goals; promote and facilitate within-team communications for ongoing reflection on the progress of the evaluation; establish processes for ongoing collaboration and dialogue between sub-study teams; the importance of a field research coordinator bridging everyday project management with scientific oversight; collect and review reflective field notes on the progress of the evaluation to aid interpretation of outcomes; and these approaches should help the identification of and reflection on possible overlaps between the evaluation and intervention.Conclusion: The lessons we have drawn point to the principle of reflexivity that, we argue, needs to become part of standard practice in the conduct of evaluations of complex interventions to promote more meaningful interpretations of the effects of an intervention and to better inform future implementation and decision-making. © 2014 Reynolds et al.; licensee BioMed Central Ltd

Discordant identification of pediatric severe sepsis by research and clinical definitions in the SPROUT international point prevalence study

Introduction: Consensus criteria for pediatric severe sepsis have standardized enrollment for research studies. However, the extent to which critically ill children identified by consensus criteria reflect physician diagnosis of severe sepsis, which underlies external validity for pediatric sepsis research, is not known. We sought to determine the agreement between physician diagnosis and consensus criteria to identify pediatric patients with severe sepsis across a network of international pediatric intensive care units (PICUs). Methods: We conducted a point prevalence study involving 128 PICUs in 26 countries across 6 continents. Over the course of 5 study days, 6925 PICU patients &lt;18 years of age were screened, and 706 with severe sepsis defined either by physician diagnosis or on the basis of 2005 International Pediatric Sepsis Consensus Conference consensus criteria were enrolled. The primary endpoint was agreement of pediatric severe sepsis between physician diagnosis and consensus criteria as measured using Cohen's ?. Secondary endpoints included characteristics and clinical outcomes for patients identified using physician diagnosis versus consensus criteria. Results: Of the 706 patients, 301 (42.6 %) met both definitions. The inter-rater agreement (? ± SE) between physician diagnosis and consensus criteria was 0.57 ± 0.02. Of the 438 patients with a physician's diagnosis of severe sepsis, only 69 % (301 of 438) would have been eligible to participate in a clinical trial of pediatric severe sepsis that enrolled patients based on consensus criteria. Patients with physician-diagnosed severe sepsis who did not meet consensus criteria were younger and had lower severity of illness and lower PICU mortality than those meeting consensus criteria or both definitions. After controlling for age, severity of illness, number of comorbid conditions, and treatment in developed versus resource-limited regions, patients identified with severe sepsis by physician diagnosis alone or by consensus criteria alone did not have PICU mortality significantly different from that of patients identified by both physician diagnosis and consensus criteria. Conclusions: Physician diagnosis of pediatric severe sepsis achieved only moderate agreement with consensus criteria, with physicians diagnosing severe sepsis more broadly. Consequently, the results of a research study based on consensus criteria may have limited generalizability to nearly one-third of PICU patients diagnosed with severe sepsis

Comparing Approaches to Research in Global and International Health: An Exploratory Study

Background: Global health is a term often used interchangeably with international health due to overlapping similarities and unclear distinctions. While some international health supporters argue that global health as a field is unnecessary as it is simply a duplicate of international health, global health supporters argue that global health is unique; for instance, it actively includes elements of empowerment and promotes cross-border collaboration. Objective: To investigate differences and similarities in research representing the fields of global and international health. Methods: We analyzed all the articles published in 2017 in two comparable academic journals representing the fields of global health (Annals of Global Health, AGH) and international health (International Health Journal, IHJ). Abstracted data included: research design and methods, income status of country of study, empowerment recommendations for practice, participation and research collaboration. Findings: Most studies in both AGH and IHJ used quantitative research methods but were significantly more common in IHJ (70%) compared to AGH (48%), whereas mores studies in AGH (17%) than IHJ (9%) used mixed methods. The majority of studies in both journals focused on low- or lower-middle income countries whereas more AGH studies (16%) focused on high-income countries compared to the IHJ studies (4%). It was more common in the AGH studies to make empowerment recommendations (90%) and to include stakeholders/users in the study (40%) compared to the IHJ studies (75% empowerment recommendations and 18% stakeholder/user participation). No difference was observed regarding cross-border research collaboration. Conclusions: This study does not show great differences between global health and international health research; however, there are still some differences indicating that global health emphasises different aspects of research compared to international health. More research is necessary to understand whether and how the distinctions between the definitions of global and international health are applied in real life, in research and beyond

Trial profile for patient exit interviews.

<p>Trial profile for patient exit interviews.</p