Strengthening nonrandomized studies of health communication strategies for HIV prevention.

BACKGROUND: There is growing interest in impact evaluations of health communication (HC) interventions for HIV prevention. Although cluster randomized trials may be optimal in terms of internal validity, they are often unfeasible for political, practical, or ethical reasons. However, a common alternative, the observational study of individuals who do and do not self-report HC intervention exposure, is prone to bias by confounding. Cluster-level quasi-experimental study designs offer promising alternatives to these extremes. METHODS: We identified common rollout strategies for HC initiatives. We mapped these scenarios against established quasi-experimental evaluation designs. We identified key issues for implementers and evaluators if these designs are to be more frequently adopted in HC intervention evaluations with high internal validity. RESULTS: Stronger evaluations will document the planned intervention components in advance of delivery and will implement interventions in clusters according to a predefined systematic allocation plan. We identify 4 types of allocation plan and their associated designs. Where some places get the HC intervention, whereas others do not, a nonrandomized controlled study may be feasible. Where HC is introduced everywhere at a defined point in time, an interrupted time series may be appropriate. Where the HC intervention is introduced in phases, a nonrandomized phased implementation or stepped-wedge design may be used. Finally, where there is variation in strength of implementation of HC, a nonrandomized, dose-response study can be planned. DISCUSSION: Our framework will assist teams planning such evaluations by identifying critical decisions for the implementers and for the evaluators of HC interventions

Statistical design and analysis plan for an impact evaluation of an HIV treatment and prevention intervention for female sex workers in Zimbabwe: a study protocol for a cluster randomised controlled trial.

BACKGROUND: Pragmatic cluster-randomised trials should seek to make unbiased estimates of effect and be reported according to CONSORT principles, and the study population should be representative of the target population. This is challenging when conducting trials amongst 'hidden' populations without a sample frame. We describe a pair-matched cluster-randomised trial of a combination HIV-prevention intervention to reduce the proportion of female sex workers (FSW) with a detectable HIV viral load in Zimbabwe, recruiting via respondent driven sampling (RDS). METHODS: We will cross-sectionally survey approximately 200 FSW at baseline and at endline to characterise each of 14 sites. RDS is a variant of chain referral sampling and has been adapted to approximate random sampling. Primary analysis will use the 'RDS-2' method to estimate cluster summaries and will adapt Hayes and Moulton's '2-step' method to adjust effect estimates for individual-level confounders and further adjust for cluster baseline prevalence. We will adapt CONSORT to accommodate RDS. In the absence of observable refusal rates, we will compare the recruitment process between matched pairs. We will need to investigate whether cluster-specific recruitment or the intervention itself affects the accuracy of the RDS estimation process, potentially causing differential biases. To do this, we will calculate RDS-diagnostic statistics for each cluster at each time point and compare these statistics within matched pairs and time points. Sensitivity analyses will assess the impact of potential biases arising from assumptions made by the RDS-2 estimation. DISCUSSION: We are not aware of any other completed pragmatic cluster RCTs that are recruiting participants using RDS. Our statistical design and analysis approach seeks to transparently document participant recruitment and allow an assessment of the representativeness of the study to the target population, a key aspect of pragmatic trials. The challenges we have faced in the design of this trial are likely to be shared in other contexts aiming to serve the needs of legally and/or socially marginalised populations for which no sampling frame exists and especially when the social networks of participants are both the target of intervention and the means of recruitment. The trial was registered at Pan African Clinical Trials Registry (PACTR201312000722390) on 9 December 2013

Re-analysis of health and educational impacts of a school-based deworming programme in western Kenya: a pure replication.

BACKGROUND: Helminth (worm) infections cause morbidity among poor communities worldwide. An influential study conducted in Kenya in 1998-99 reported that a school-based drug-and-educational intervention had benefits regarding worm infections and school attendance. Effects were seen among children treated with deworming drugs, untreated children in intervention schools and children in nearby non-intervention schools. Combining these effects, the intervention was reported to increase school attendance by 7.5% in treated children. Effects on other outcomes (worm infections, anaemia, nutritional status and examination performance) were also investigated. METHODS: In this pure replication, we used data provided by the original authors to re-analyse the study according to their methods. We compared these results against those presented in the original paper. RESULTS: Although most results were reproduced as originally reported, we identified discrepancies of several types between the original findings and re-analysis. For worm infections, re-analysis showed reductions similar to those originally reported. For anaemia prevalence, in contrast to the original findings, re-analysis found no evidence of benefit. For nutritional status, both original findings and re-analysis described modest evidence for a small improvement. For school attendance, re-analysis showed benefits similar to those originally found in intervention schools for both children who did and those who did not receive deworming drugs. However, after correction of coding errors, there was little evidence of an indirect effect on school attendance among children in schools close to intervention schools. Combining these effects gave a total increase in attendance of 3.9% among treated children, which was no longer statistically significant. As in the original results, re-analysis found no effect of the intervention on examination performance. CONCLUSIONS: Re-applying analytical approaches originally used, but correcting various errors, we found little evidence for some previously-reported indirect effects of a deworming intervention. Effects on worm infections, nutritional status, examination performance and school attendance on children in intervention schools were largely unchanged

Competition is not necessarily a barrier to community mobilisation among sex workers: an intervention planning assessment from Zimbabwe.

BACKGROUND: Community mobilization among female sex workers (SWs) is recognized as an effective strategy to empower SWs and increase their uptake of health services. Activities focus on increasing social cohesion between SWs by building trust, strengthening networks, and encouraging shared efforts for mutual gain. Several studies, however, suggest that high levels of interpersonal competition between SWs can pose a barrier to collective action and support. We conducted a study to examine levels of perceived competition between SWs in Mutare, Hwange and Victoria Falls in Zimbabwe in order to inform development of a community-based intervention for HIV prevention and treatment. This paper focuses on our qualitative findings and their implications for the design of HIV programming in the Zimbabwean context. METHODS: Following a respondent driven sampling (RDS) survey, we explored issues related to social cohesion amongst SWs in Mutare, Hwange and Victoria Falls through in-depth interviews conducted with 22 SWs. Interviews examined dynamics of SWs' relationships and extent of social support, and were analyzed using thematic content analysis using the constant comparative method. Findings are contextualised against descriptive data extracted from the survey, which was analysed using Stata 12, adjusting for RDS. RESULTS: Across all sites, women described protecting each other at night, advising each other about violent or non-paying clients, and paying fines for each other following arrest. In Mutare, women gave additional examples, including physically attacking problem clients, treatment adherence support and shared saving schemes. However, interviews also highlighted fierce competition between women and deep mistrust. This reflects the reported mix of competition and support from the survey of 836 women (Mutare n = 370, Hwange n = 237, Victoria Falls n = 229). In Mutare, 92.8 % of SWs agreed there was a lot of competition; 87.9 % reported that SWs support each other. This contrasted with Victoria Falls and Hwange where fewer agreed there was competition between SWs (70.5 % and 78.0 %), but also fewer reported that SWs support each other at work (55.2 % and 51.2 %). CONCLUSIONS: Women reported being most likely to support each other when confronted with serious danger but maintained high levels of competition for clients, suggesting competition at work does not represent a barrier to support. Examples of practical assistance between SWs provide entry points for our planned community mobilization activities, which aim to broaden trust and support among SWs while acknowledging their professional competition

Re-analysis of health and educational impacts of a school-based deworming programme in western Kenya: a statistical replication of a cluster quasi-randomized stepped-wedge trial.

INTRODUCTION: Helminth (worm) infections cause morbidity among poor communities worldwide. An influential study conducted in Kenya in 1998-99 reported that a school-based drug-and-educational intervention had benefits for worm infections and school attendance. METHODS: In this statistical replication, we re-analysed data from this cluster quasi-randomized stepped-wedge trial, specifying two co-primary outcomes: school attendance and examination performance. We estimated intention-to-treat effects using year-stratified cluster-summary analysis and observation-level random-effects regression, and combined both years with a random-effects model accounting for year. The participants were not blinded to allocation status, and other interventions were concurrently conducted in a sub-set of schools. A protocol guiding outcome data collection was not available. RESULTS: Quasi-randomization resulted in three similar groups of 25 schools. There was a substantial amount of missing data. In year-stratified cluster-summary analysis, there was no clear evidence for improvement in either school attendance or examination performance. In year-stratified regression models, there was some evidence of improvement in school attendance [adjusted odds ratios (aOR): year 1: 1.48, 95% confidence interval (CI) 0.88-2.52, P = 0.147; year 2: 1.23, 95% CI 1.01-1.51, P = 0.044], but not examination performance (adjusted differences: year 1: -0.135, 95% CI -0.323-0.054, P = 0.161; year 2: -0.017, 95% CI -0.201-0.166, P = 0.854). When both years were combined, there was strong evidence of an effect on attendance (aOR 1.82, 95% CI 1.74-1.91, P < 0.001), but not examination performance (adjusted difference -0.121, 95% CI -0.293-0.052, P = 0.169). CONCLUSIONS: The evidence supporting an improvement in school attendance differed by analysis method. This, and various other important limitations of the data, caution against over-interpretation of the results. We find that the study provides some evidence, but with high risk of bias, that a school-based drug-treatment and health-education intervention improved school attendance and no evidence of effect on examination performance

Designing a stepped wedge trial: three main designs, carry-over effects and randomisation approaches.

BACKGROUND: There is limited guidance on the design of stepped wedge cluster randomised trials. Current methodological literature focuses mainly on trials with cross-sectional data collection at discrete times, yet many recent stepped wedge trials do not follow this design. In this article, we present a typology to characterise the full range of stepped wedge designs, and offer guidance on several other design aspects. METHODS: We developed a framework to define and report the key characteristics of a stepped wedge trial, including cluster allocation and individual participation. We also considered the relative strengths and weaknesses of trials according to this framework. We classified recently published stepped wedge trials using this framework and identified illustrative case studies. We identified key design choices and developed guidance for each. RESULTS: We identified three main stepped wedge designs: those with a closed cohort, an open cohort, and a continuous recruitment short exposure design. In the first two designs, many individuals experience both control and intervention conditions. In the final design, individuals are recruited in continuous time as they become eligible and experience either the control or intervention condition, but not both, and then provide an outcome measurement at follow-up. While most stepped wedge trials use simple randomisation, stratification and restricted randomisation are often feasible and may be useful. Some recent studies collect outcome information from individuals exposed a long time before or after the rollout period, but this contributes little to the primary analysis. Incomplete designs should be considered when the intervention cannot be implemented quickly. Carry-over effects can arise in stepped wedge trials with closed and open cohorts. CONCLUSIONS: Stepped wedge trial designs should be reported more clearly. Researchers should consider the use of stratified and/or restricted randomisation. Trials should generally not commit resources to collect outcome data from individuals exposed a long time before or after the rollout period. Though substantial carry-over effects are uncommon in stepped wedge trials, researchers should consider their possibility before conducting a trial with closed or open cohorts

Measuring implementation strength: lessons from the evaluation of public health strategies in low- and middle-income settings.

Evaluation of strategies to ensure evidence-based, low-cost interventions reach those in need is critical. One approach is to measure the strength, or intensity, with which packages of interventions are delivered, in order to explore the association between implementation strength and public health gains. A recent systematic review suggested methodological guidance was needed. We described the approaches used in three examples of measures of implementation strength in evaluation. These addressed important public health topics with a substantial disease burden in low-and middle-income countries; they involved large-scale implementation; and featured evaluation designs without comparison areas. Strengths and weaknesses of the approaches were discussed. In the evaluation of Ethiopia's Health Extension Programme, implementation strength scoring for each kebele (ward) was based on aggregated data from interviews with mothers of children aged 12-23 months, reflecting their reports of contact with four elements of the programme. An evaluation of the Avahan HIV prevention programme in India used the cumulative amount of Avahan funding per HIV-infected person spent each year in each district. In these cases, a single measure was developed and the association with hypothesised programme outcomes presented. In the evaluation of the Affordable Medicines Facility-malaria, several implementation strength measures were developed based on the duration of activity of the programme and the level of implementation of supporting interventions. Measuring the strength of programme implementation and assessing its association with outcomes is a promising approach to strengthen pragmatic impact evaluation. Five key aspects of developing an implementation strength measure are to: (a) develop a logic model; (b) identify aspects of implementation to be assessed; (c) design and implement data collection from a range of data sources; (d) decide whether and how to combine data into a single measure; and, (e) plan whether and how to use the measure(s) in outcome analysis

Trials and tribulations: cross-learning from the practices of epidemiologists and economists in the evaluation of public health interventions.

The randomized controlled trial is commonly used by both epidemiologists and economists to test the effectiveness of public health interventions. Yet we have noticed differences in practice between the two disciplines. In this article, we propose that there are some underlying differences between the disciplines in the way trials are used, how they are conducted and how results from trials are reported and disseminated. We hypothesize that evidence-based public health could be strengthened by understanding these differences, harvesting best-practice across the disciplines and breaking down communication barriers between economists and epidemiologists who conduct trials of public health interventions

Supporting adolescent girls to stay in school, reduce child marriage and reduce entry into sex work as HIV risk prevention in north Karnataka, India: protocol for a cluster randomised controlled trial.

BACKGROUND: Low caste adolescent girls living in rural northern Karnataka are at increased risk of school drop-out, child marriage, and entry into sex-work, which enhances their vulnerability to HIV, early pregnancy and adverse maternal and child health outcomes. This protocol describes the evaluation of Samata, a comprehensive, multi-level intervention designed to address these structural drivers of HIV risk and vulnerability. METHODS/DESIGN: The Samata study is a cluster randomised controlled trial that will be conducted in eighty village clusters (40 intervention; 40 control) in Bijapur and Bagalkot districts in northern Karnataka. The intervention seeks to reach low caste girls and their families; adolescent boys; village communities; high school teachers and school governing committees; and local government officials. All low caste (scheduled caste/tribe) adolescent girls attending 7th standard (final year of primary school) will be recruited into the study in two consecutive waves, one year apart. Girls (n = 2100), their families (n = 2100) and school teachers (n = 650) will be interviewed at baseline and at endline. The study is designed to assess the impact of the intervention on four primary outcomes: the proportion of low caste girls who (i) enter into secondary school; (ii) complete secondary school; (iii) marry before age 15; and (iv) engage in sex before age 15. Observers assessing the outcomes will be blinded to group assignment. The primary outcome will be an adjusted, cluster-level intention to treat analysis, comparing outcomes in intervention and control villages at follow-up. We will also conduct survival analyses for the following secondary outcomes: marriage, sexual debut, pregnancy and entry into sex work. Complementary monitoring and evaluation, qualitative and economic research will be used to explore and describe intervention implementation, the pathways through which change occurs, and the cost-effectiveness of the intervention. DISCUSSION: This is an innovative trial of a comprehensive intervention to improve the quality of life and reduce HIV vulnerability among marginalised girls in northern Karnataka. The findings will be of interest to programme implementers, policy makers and evaluation researchers working in the development, education, and sexual and reproductive health fields. TRIAL REGISTRATION: ClinicalTrials.Gov NCT01996241 . 16th November 2013