Strategies to improve retention in randomised trials: a Cochrane systematic review and meta-analysis

Objective: To quantify the effect of strategies to improve retention in randomised trials.<p></p> Design: Systematic review and meta-analysis.<p></p> Data sources Sources searched: MEDLINE, EMBASE, PsycINFO, DARE, CENTRAL, CINAHL, C2-SPECTR, ERIC, PreMEDLINE, Cochrane Methodology Register, Current Controlled Trials metaRegister, WHO trials platform, Society for Clinical Trials (SCT) conference proceedings and a survey of all UK clinical trial research units.<p></p> Review: methods Included trials were randomised evaluations of strategies to improve retention embedded within host randomised trials. The primary outcome was retention of trial participants. Data from trials were pooled using the fixed-effect model. Subgroup analyses were used to explore the heterogeneity and to determine whether there were any differences in effect by the type of strategy.<p></p> Results: 38 retention trials were identified. Six broad types of strategies were evaluated. Strategies that increased postal questionnaire responses were: adding, that is, giving a monetary incentive (RR 1.18; 95% CI 1.09 to 1.28) and higher valued incentives (RR 1.12; 95% CI 1.04 to 1.22). Offering a monetary incentive, that is, an incentive given on receipt of a completed questionnaire, also increased electronic questionnaire response (RR 1.25; 95% CI 1.14 to 1.38). The evidence for shorter questionnaires (RR 1.04; 95% CI 1.00 to 1.08) and questionnaires relevant to the disease/condition (RR 1.07; 95% CI 1.01 to 1.14) is less clear. On the basis of the results of single trials, the following strategies appeared effective at increasing questionnaire response: recorded delivery of questionnaires (RR 2.08; 95% CI 1.11 to 3.87); a ‘package’ of postal communication strategies (RR 1.43; 95% CI 1.22 to 1.67) and an open trial design (RR 1.37; 95% CI 1.16 to 1.63). There is no good evidence that the following strategies impact on trial response/retention: adding a non-monetary incentive (RR=1.00; 95% CI 0.98 to 1.02); offering a non-monetary incentive (RR=0.99; 95% CI 0.95 to 1.03); ‘enhanced’ letters (RR=1.01; 95% CI 0.97 to 1.05); monetary incentives compared with offering prize draw entry (RR=1.04; 95% CI 0.91 to 1.19); priority postal delivery (RR=1.02; 95% CI 0.95 to 1.09); behavioural motivational strategies (RR=1.08; 95% CI 0.93 to 1.24); additional reminders to participants (RR=1.03; 95% CI 0.99 to 1.06) and questionnaire question order (RR=1.00, 0.97 to 1.02). Also based on single trials, these strategies do not appear effective: a telephone survey compared with a monetary incentive plus questionnaire (RR=1.08; 95% CI 0.94 to 1.24); offering a charity donation (RR=1.02, 95% CI 0.78 to 1.32); sending sites reminders (RR=0.96; 95% CI 0.83 to 1.11); sending questionnaires early (RR=1.10; 95% CI 0.96 to 1.26); longer and clearer questionnaires (RR=1.01, 0.95 to 1.07) and participant case management by trial assistants (RR=1.00; 95% CI 0.97 to 1.04).<p></p> Conclusions: Most of the trials evaluated questionnaire response rather than ways to improve participants return to site for follow-up. Monetary incentives and offers of monetary incentives increase postal and electronic questionnaire response. Some strategies need further evaluation. Application of these results would depend on trial context and follow-up procedures.<p></p&gt

Identifying additional studies for a systematic review of retention strategies in randomised controlled trials: making contact with trials units and trial methodologists

BACKGROUND: Search strategies for systematic reviews aim to identify all evidence relevant to the research question posed. Reports of methodological research can be difficult to find leading to biased results in systematic reviews of research methodology. Evidence suggests that contact with investigators can help to identify unpublished research. To identify additional eligible randomised controlled trials (RCTs) for a Cochrane systematic review of strategies to improve retention in RCTs, we conducted a survey of UK clinical trials units (CTUs) and made contact with RCT methodologists. METHODS: Key contacts for all UK CTUs were sent a personalised email with a short questionnaire and summary protocol of the Cochrane methodology review. The questionnaire asked whether a RCT evaluating strategies to improve retention embedded in a RCT had ever been conducted by the CTU. Questions about the stage of completion and publication of such RCTs were included. The summary protocol outlined the aims, eligibility criteria, examples of types of retention strategies, and the primary outcome for the systematic review. Personal communication with RCT methodologists and presentations of preliminary results of the review at conferences were also used to identify additional eligible RCTs. We checked the results of our standard searches to see if eligible studies identified through these additional methods were also found using our standard searches. RESULTS: We identified 14 of the 38 RCTs included in the Cochrane methodology review by contacting trials units and methodologists. Eleven of the 14 RCTs identified by these methods were either published in grey literature, in press or unpublished. Three remaining RCTs were fully published at the time. Six of the RCTs identified were not found through any other searches. The RCTs identified represented data for 6 of 14 RCTs of incentive strategies (52% of randomised participants included in the review), and 6 of 14 RCTs of communication strategies (52% of randomised participants included in the Cochrane review). Data were unavailable for two of the RCTs identified. CONCLUSIONS: Methodological evaluations embedded in RCTs may be unpublished, published in the grey literature or where published, poorly indexed in bibliographic databases. To identify such studies and minimise selection bias in systematic reviews of methodological evaluations, reviewers should consider contacting CTUs and trial methodologists

Best practice guidance for the use of strategies improved retention in randomised trials developed from two consensus workshops

OBJECTIVE: To develop best practice guidance for the use of retention strategies in randomised clinical trials (RCTs). DESIGN: Consensus development workshops. SETTING: Two UK Clinical Trials Units. PARTICIPANTS: 66 statisticians, clinicians, RCT coordinators, research scientists, research assistants, and data managers associated with RCTs. METHODS: The consensus development workshops were based on the consensus development conference method used to develop best practice for treatment of medical conditions. Workshops commenced with a presentation of the evidence for: incentives, communication, questionnaire format, behavioural, case management and methodological retention strategies identified by a Cochrane review and associated qualitative study. Three simultaneous group discussions followed, focused on: a) how convinced the workshop participants were by the evidence for retention strategies, b) barriers to the use of effective retention strategies, c) types of RCT follow-up that retention strategies could be used for, and d) strategies for future research. Summaries of each group discussion were fed back to the workshop. Coded content for both workshops were compared for agreement and disagreement. Agreed consensus on best practice guidance for retention was identified. RESULTS: Workshop participants agreed best practice guidance for the use of small financial incentives to improve response to postal questionnaires in RCTs. Use of 2nd class post was thought to be adequate for postal communication with RCT participants. The most relevant validated questionnaire was considered best practice for collecting RCT data. Barriers identified for the use of effective retention strategies were: the small improvements seen in questionnaire response for the addition of monetary incentives, and perceptions among trialists that some communication strategies are outdated. Furthermore, there was resistance to change existing retention practices thought to be effective. Face to face and electronic follow-up technologies were identified as retention strategies for further research. CONCLUSIONS: We developed best practice guidance for the use of retention strategies in RCTs and identified potential barriers to the use of effective strategies. The extent of agreement on best practice is limited by the variability in the currently available evidence. This guidance will need updating as new retention strategies are developed and evaluated

Use of strategies to improve retention in primary care randomised trials: a qualitative study with in-depth interviews

Objective To explore the strategies used to improve retention in primary care randomised trials.<p></p> Design Qualitative in-depth interviews and thematic analysis.<p></p> Participants 29 UK primary care chief and principal investigators, trial managers and research nurses.<p></p> Methods In-depth face-to-face interviews.<p></p> Results Primary care researchers use incentive and communication strategies to improve retention in trials, but were unsure of their effect. Small monetary incentives were used to increase response to postal questionnaires. Non-monetary incentives were used although there was scepticism about the impact of these on retention. Nurses routinely used telephone communication to encourage participants to return for trial follow-up. Trial managers used first class post, shorter questionnaires and improved questionnaire designs with the aim of improving questionnaire response. Interviewees thought an open trial design could lead to biased results and were negative about using behavioural strategies to improve retention. There was consensus among the interviewees that effective communication and rapport with participants, participant altruism, respect for participant's time, flexibility of trial personnel and appointment schedules and trial information improve retention. Interviewees noted particular challenges with retention in mental health trials and those involving teenagers.<p></p> Conclusions The findings of this qualitative study have allowed us to reflect on research practice around retention and highlight a gap between such practice and current evidence. Interviewees describe acting from experience without evidence from the literature, which supports the use of small monetary incentives to improve the questionnaire response. No such evidence exists for non-monetary incentives or first class post, use of which may need reconsideration. An exploration of barriers and facilitators to retention in other research contexts may be justified.<p></p&gt

Primary care morbidity in Eastern Cape Province

Background. Primary health care in rural South Africa is predominantly provided by remote clinics and health centres. In 1994, health centres were upgraded and new health centres developed to serve as a health care filter between community clinics and district hospitals.Aim. To describe the spectrum of clinical problems encountered at a new health centre in an area of high economic deprivation and compare this with an adjacent community clinic and district hospital.Design. Cross-sectional survey.Setting. A rural clinic, health centre and district hospital in Eastern Cape Province, South Africa.Methods. The International Classification of Primary Care-2(ICPC-2) was used to code data collected over a 13-week period from patients presenting at a community clinic, health centre and district hospital.Results. Altogether, 4 383 patient encounters were recorded across all three sites. Most contacts at the clinic (97%) and the health centre (80%) were with a nurse. Females over 15 years of age comprised over half of all contacts at health facilities (53%). The most common diagnosis category was respiratory (23%). Cough was the most common symptom.Thirty per cent of children up to 5 years of age were seen for immunisations. Most childhood immunisations (79%) werecarried out at the health centre.Conclusion. Of all the health care facilities surveyed, the health centre had the highest throughput of patients, indicating that the health centre is an efficient filter between the community and hospital. The ICPC-2 can be successfully used to monitor encounters at similar African health care facilities.S Afr Med J 2010; 100: 309-312

Increasing follow-up questionnaire response rates in a randomized controlled trial of telehealth for depression: three embedded controlled studies

Background: Attrition is problematic in trials, and may be exacerbated in longer studies, telehealth trials and participants with depression – three features of The Healthlines Study. Advance notification, including a photograph and using action-oriented email subject lines might increase response rates, but require further investigation. We examined the effectiveness of these interventions in three embedded Healthlines studies. Methods: Based in different trial sites, participants with depression were alternately allocated to be pre-called or not ahead of the 8-month follow-up questionnaire (Study 1), randomized to receive a research team photograph or not with their 12-month questionnaire (Study 2), and randomized to receive an action-oriented (‘ACTION REQUIRED’) or standard (‘Questionnaire reminder’) 12-month email reminder (Study 3). Participants could complete online or postal questionnaires, and received up to five questionnaire reminders. The primary outcome was completion of the Patient Health Questionnaire (PHQ-9). Secondary outcome measures were the number of reminders and time to questionnaire completion. Results: Of a total of 609 Healthlines depression participants, 190, 251 and 231 participants were included in Studies 1–3 (intervention: 95, 126 and 115), respectively. Outcome completion was ≥90 % across studies, with no differences between trial arms (Study 1: OR 0.38, 95 % CI 0.07–2.10; Study 2: OR 0.84, 95 % CI 0.26–2.66; Study 3: OR 0.53 95 % CI 0.19–1.49). Pre-called participants were less likely to require a reminder (48.4 % vs 62.1 %, OR 0.41, 95 % CI 0.21–0.78), required fewer reminders (adjusted difference in means −0.67, 95 % CI −1.13 to −0.20), and completed follow-up quicker (median 8 vs 15 days, HR 1.35, 95 % CI 1.00–1.82) than control subjects. There were no significant between-group differences in Studies 2 or 3. Conclusions: Eventual response rates in this trial were high, with no further improvement from these interventions. While the photograph and email interventions were ineffective, pre-calling participants reduced time to completion. This strategy might be helpful when the timing of study completion is important. Researchers perceived a substantial benefit from the reduction in reminders with pre-calling, despite no overall decrease in net effort after accounting for pre-notification

Including a pen and/or cover letter, containing social incentive text, had no effect on questionnaire response rate : a factorial randomised controlled Study within a Trial [version 1; peer review: awaiting peer review]

Background: Postal questionnaires are frequently used in randomised controlled trials to collect outcome data on participants; however, poor response can introduce bias, affect generalisability and validity, and reduce statistical power. The objective of this study was to assess whether a pen and/or social incentive text cover letter sent with a postal follow-up questionnaire increased response rates in a trial. Method: A two-by-two factorial randomised controlled trial was embedded within the OTIS host trial. Participants due their 12-month (final) follow-up questionnaire were randomised to be sent: a pen; a social incentive text cover letter; both; or neither. The primary outcome measure was the proportion of participants in each group who returned the questionnaire. Secondary outcomes were: time to return, completeness of the questionnaire, necessity of a reminder letter, and the cost effectiveness. Results: The overall 12-month questionnaire response rate was 721 out of 755 (95.5%). Neither the pen nor social incentive cover letter had a statistically significant effect on response rate: pen 95.2% vs. no pen 95.8%, adjusted OR 0.90 (95% CI 0.45 to 1.80; p=0.77); social incentive cover letter 95.2% vs. no social incentive cover letter 95.8%, adjusted OR 0.84 (95% CI 0.42 to 1.69, p=0.63). No statistically significant differences were observed between either of the intervention groups on time to response, need for a reminder or completeness. Therefore, neither intervention was cost-effective. Conclusions: We found no evidence of a difference in response rates associated with the inclusion of a pen and/or social incentive cover letter with the final follow-up postal questionnaire of the host trial. However, when these results are combined with previous SWATs, the meta-analysis evidence remains that including a pen increases response rates. The social incentive cover letter warrants further investigation to determine effectiveness

Systematic techniques for assisting recruitment to trials (START): study protocol for embedded, randomized controlled trials

BACKGROUND: Randomized controlled trials play a central role in evidence-based practice, but recruitment of participants, and retention of them once in the trial, is challenging. Moreover, there is a dearth of evidence that research teams can use to inform the development of their recruitment and retention strategies. As with other healthcare initiatives, the fairest test of the effectiveness of a recruitment strategy is a trial comparing alternatives, which for recruitment would mean embedding a recruitment trial within an ongoing host trial. Systematic reviews indicate that such studies are rare. Embedded trials are largely delivered in an ad hoc way, with interventions almost always developed in isolation and tested in the context of a single host trial, limiting their ability to contribute to a body of evidence with regard to a single recruitment intervention and to researchers working in different contexts. METHODS/DESIGN: The Systematic Techniques for Assisting Recruitment to Trials (START) program is funded by the United Kingdom Medical Research Council (MRC) Methodology Research Programme to support the routine adoption of embedded trials to test standardized recruitment interventions across ongoing host trials. To achieve this aim, the program involves three interrelated work packages: (1) methodology - to develop guidelines for the design, analysis and reporting of embedded recruitment studies; (2) interventions - to develop effective and useful recruitment interventions; and (3) implementation - to recruit host trials and test interventions through embedded studies. DISCUSSION: Successful completion of the START program will provide a model for a platform for the wider trials community to use to evaluate recruitment interventions or, potentially, other types of intervention linked to trial conduct. It will also increase the evidence base for two types of recruitment intervention. TRIAL REGISTRATION: The START protocol covers the methodology for embedded trials. Each embedded trial is registered separately or as a substudy of the host trial

What are the most important unanswered research questions in trial retention? A James Lind Alliance Priority Setting Partnership: the PRioRiTy II (Prioritising Retention in Randomised Trials) study

Background One of the top three research priorities for the UK clinical trial community is to address the gap in evidence-based approaches to improving participant retention in randomised trials. Despite this, there is little evidence supporting methods to improve retention. This paper reports the PRioRiTy II project, a Priority Setting Partnership (PSP) that identified and prioritised unanswered questions and uncertainties around trial retention in collaboration with key stakeholders. Methods This PSP was conducted in collaboration with the James Lind Alliance, a non-profit making initiative, to support key stakeholders (researchers, patients, and the public) in jointly identifying and agreeing on priority research questions. There were three stages. (1) First an initial online survey was conducted consisting of six open-ended questions about retention in randomised trials. Responses were coded into thematic groups to create a longlist of questions. The longlist of questions was checked against existing evidence to ensure that they had not been answered by existing research. (2) An interim stage involved a further online survey where stakeholders were asked to select questions of key importance from the longlist. (3) A face-to-face consensus meeting was held, where key stakeholder representatives agreed on an ordered list of 21 unanswered research questions for methods of improving retention in randomised trials. Results A total of 456 respondents yielded 2431 answers to six open-ended questions, from which 372 questions specifically about retention were identified. Further analysis included thematically grouping all data items within answers and merging questions in consultation with the Steering Group. This produced 27 questions for further rating during the interim survey. The top 21 questions from the interim online survey were brought to a face-to-face consensus meeting in which key stakeholder representatives prioritised the order. The ‘Top 10’ of these are reported in this paper. The number one ranked question was ’What motivates a participant’s decision to complete a clinical trial?’ The entire list will be available at www.priorityresearch.ie. Conclusion The Top 10 list can inform the direction of future research on trial methods and be used by funders to guide projects aiming to address and improve retention in randomised trials