No difference found in time to publication by statistical significance of trial results: a methodological review

Objective: Time-lag from study completion to publication is a potential source of publication bias in randomised controlled trials. This study sought to update the evidence base by identifying the effect of the statistical significance of research findings on time to publication of trial results.Design: Literature searches were carried out in four general medical journals from June 2013 to June 2014 inclusive (BMJ, JAMA, the Lancet and the New England Journal of Medicine). Setting: Methodological review of four general medical journals. Participants: Original research articles presenting the primary analyses from phase 2, 3 and 4 parallel-group randomised controlled trials were included. Main outcome measures: Time from trial completion to publication.Results: The median time from trial completion to publication was 431 days (n ¼ 208, interquartile range 278–618). A multivariable adjusted Cox model found no statistically significant difference in time to publication for trials reporting positive or negative results (hazard ratio: 0.86, 95% CI 0.64 to 1.16, p ¼ 0.32). Conclusion: In contrast to previous studies, this review did not demonstrate the presence of time-lag bias in time to publication. This may be a result of these articles being published in four high-impact general medical journals that may be more inclined to publish rapidly, whatever the findings. Further research is needed to explore the presence of time-lag bias in lower quality studies and lower impact journals

Pragmatic cluster randomised controlled trial of contextualised grammar teaching and small group teaching to improve the writing skills of 11 year old children

Introduction: We evaluated two interventions: a contextualised grammar teaching intervention – Grammar for Writing - to assess whether it improved 11 year old children’s writing skills; and a small group literacy intervention to assess whether or not this was effective. Design and method: We used a pragmatic cluster randomised trial with partial split plot design. Independent concealed randomisation was undertaken at the class level, and, within the intervention group, children were also individually randomised to receive the whole class intervention plus a small group intervention or to receive the intervention in a whole class setting only. The main outcomes were writing and reading assessed by the Progress in English 11 (Long Form) test (GL Assessment). Results: In 2013, 55 schools in England, each with two classes, were recruited and randomised. Within each school, the two classes were randomly allocated to receive either the intervention or the control condition. After randomisation, 2 schools withdrew, leaving 53 schools, 106 classes and 2510 pupils. We observed an effect size (ES) of 0.10 favouring the Grammar for Writing classes; however, this was not statistically significant (95% confidence interval (CI) -0.10 to 0.31). Pupils randomised to the small groups had an increased literacy score when compared with the control classes (ES = 0.24, 95% CI 0.00 to 0.49) and when compared with the intervention children taught in the whole class (ES = 0.21, 95% CI 0.04 to 0.38). Conclusion: There is little evidence that this form of contextualised grammar teaching had an effect on 11 year old children’s writing skills. There was some evidence of an effect for small group teaching

Qualitative research to inform hypothesis testing for fidelity-based sub-group analysis in clinical trials : lessons learnt from the process evaluation of a multifaceted podiatry intervention for falls prevention

Background: Ensuring fidelity to complex interventions is a challenge when conducting pragmatic randomised controlled trials. We explore fidelity through a qualitative process evaluation, which was conducted alongside a pragmatic, multicentre, two-arm cohort randomised controlled trial: the REFORM (Reducing Falls with Orthoses and a Multifaceted podiatry intervention) trial. The paper aims, through a qualitative process evaluation, to explore some of the factors that may have affected the delivery of the REFORM intervention and highlight how project-specific fidelity can be assessed using a truly mixed-methods approach when informed by qualitative insights. Design: Semi-structured qualitative interviews carried out as part of a process evaluation. Interviews were analysed thematically. Setting: Seven NHS trusts in the UK and a University podiatry school in Ireland. Interviews were undertaken face-toface or over the telephone. Participants: Twenty-one REFORM trial participants and 14 podiatrists who delivered the REFORM intervention. Results: Factors affecting fidelity included: how similar the intervention was to routine practice; the challenges of delivering a multifaceted intervention to a heterogeneous older population; and practical issues with delivery such as time and training. Trial participants’ views of the intervention, whether falls prevention is a personal priority, their experience of being part of a trial and individual factors such as medical conditions may also have affected intervention fidelity. Conclusions: Our process evaluation highlighted factors that were perceived to have affected the fidelity of the REFORM intervention and in doing so demonstrates the importance of considering fidelity when designing and evaluating pragmatic trials. We propose a number of recommendations of how important project-specific insights from qualitative work can be incorporated into the design of fidelity measurement of future trials, which build on existing conceptual fidelity frameworks. In particular, we encourage adopting a mixed-methods approach whereby qualitative insights can be used to suggest ways to enhance quantitative data collection facilitating integration through hypothesis generation, hypothesis testing and seeking explanation for trial findings. This will provide a framework of enabling measures of fidelity to be incorporated into the understanding of trial results which has been relatively neglected by existing literature. Trial registration: ISRCTN Registry: ISRCTN68240461. Registered on 01/07/2011. Keywords: Process evaluation, Randomised controlled trials, Fidelity, Mixed methods, Falls, Elderly, Ageing, Qualitativ

Exploring Engagement with Authors of Randomised Controlled Trials to Develop Recommendations to Improve Allocation Concealment Implementation and Reporting

Background: Reviews have consistently shown that allocation concealment is frequently implemented and reported suboptimally in randomised controlled trials (RCTs). This research aims to pilot engaging with authors of RCTs to explore their knowledge and understanding of allocation concealment implementation and reporting to ascertain areas and mechanisms for their improvement. Methods: Authors that published RCTs in core clinical journals in one month in 2019 were identified. Authors were invited to complete questionnaires to elicit their views and experiences on the implementation and reporting quality focussing on allocation concealment. Methodological quality of allocation concealment was evaluated in this sample by assessing adherence to the Consolidated Standards of Reporting Trials (CONSORT). Results: Reporting was suboptimal, with only 57% of allocation concealment methods reported to be implemented which were judged as adequate, with 18% using sealed envelopes and more than 40% not adequately reporting allocation methods. When exploring allocation concealment, implementation and reporting questionnaires were found to elicit a low response rate amongst authors of RCTs. Discussion: Following analysis of the themes that emerged from the questionnaires, the main recommendations to improve reporting quality are: journals need to endorse, adhere and promote reporting guidelines, a methodologist could review methodological details of publications simultaneously to peer review, envelopes as a form of allocation concealment are poorly implemented and reported, so careful review of these is required, funders need to insist on more robust allocation concealment methods are employed if the RCT setting allows, and authors need to acknowledge their responsibility for transparent reporting of RCTs

Timing of electronic reminders did not improve trial participant questionnaire response: a randomized trial and meta-analyses

Objectives: The objective of this study was to assess whether timing of short messaging service (SMS) reminders improved postal questionnaire return rates from participants in a randomized controlled trial (RCT). Study Design and Setting: A Study Within A Trial (SWAT) embedded in a multicenter RCT evaluating three treatments for the frozen shoulder. Participants who provided a mobile telephone number were randomized to either prenotification SMS on the day of the questionnaire mail-out or postnotification SMS 4 days after questionnaire mail-out for the 3-month follow-up. The primary outcome was the proportion of participants who returned a valid questionnaire. A systematic review was undertaken to identify other embedded trials to perform a meta-analysis. Results: Of the 269 participants, 122/135 (90.4%) returned a valid questionnaire in the prenotification arm and 119/134 (88.8%) in the postnotification arm (difference of −1.6%; 95% CI of difference: −8.9%, 5.7%). There was no difference in time to response (HR = 1.04; 95% CI: 0.80 to 1.34) or need for additional reminders (OR = 0.71; 95% CI: 0.43 to 1.17). Meta-analysis of two RCTs showed no difference in response rates between prenotification and postnotification reminders (OR = 0.78 95% CI: 0.42 to 1.45). Conclusion: Timing of SMS reminders did not improve response rates and time to response or affect the need for additional reminders

Vitamin D with Calcium reduces mortality: patient level pooled analysis of 70,528 patients from eight major vitamin D trials

Introduction: Vitamin D may affect multiple health outcomes. If so, an effect on mortality is to be expected. Using pooled data from randomized controlled trials, we performed individual patient data (IPD) and trial level meta-analyses to assess mortality among participants randomized to either vitamin D alone or vitamin D with calcium. Subjects and Methods: Through a systematic literature search, we identified 24 randomized controlled trials reporting data on mortality in which vitamin D was given either alone or with calcium. From a total of 13 trials with more than 1000 participants each, eight trials were included in our IPD analysis. Using a stratified Cox regression model, we calculated risk of death during 3 yr of treatment in an intention-to-treat analysis. Also, we performed a trial level meta-analysis including data from all studies. Results: The IPD analysis yielded data on 70,528 randomized participants (86.8% females) with a median age of 70 (interquartile range, 62–77) yr. Vitamin D with or without calcium reduced mortality by 7% [hazard ratio, 0.93; 95% confidence interval (CI), 0.88–0.99]. However, vitamin D alone did not affect mortality, but risk of death was reduced if vitamin D was given with calcium (hazard ratio, 0.91; 95% CI, 0.84–0.98). The number needed to treat with vitamin D plus calcium for 3 yr to prevent one death was 151. Trial level meta-analysis (24 trials with 88,097 participants) showed similar results, i.e. mortality was reduced with vitamin D plus calcium (odds ratio, 0.94; 95% CI, 0.88–0.99), but not with vitamin D alone (odds ratio, 0.98; 95% CI, 0.91–1.06). Conclusion: Vitamin D with calcium reduces mortality in the elderly, whereas available data do not support an effect of vitamin D alone

Undertaking Studies Within A Trial to evaluate recruitment and retention strategies for randomised controlled trials : lessons learnt from the PROMETHEUS research programme

Funding: This award was funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme (NIHR award ref: 13/55/80) and is published in full in Health Technology Assessment; Vol. 28, No. 2. See the NIHR Funding and Awards website for further award information.Peer reviewedPublisher PD

Trial Forge Guidance 4 : A guideline for reporting the results of randomised Studies Within A Trial (SWATs)

Acknowledgements We would like to thank those members of the PROMETHEUS programme who are not explicitly named as authors of this publication, but who provided valuable input to the delivery and conduct of the programme within which the guideline development sat. PROMETHEUS programme members include co-authors CA, LK, AP, DB, CC, DD, SG, KG, CH, CS, DT, and ST and P Bower (University of Manchester), L Culliford (University of Bristol), L Doherty (University of York), and R Emsley (Kings College London). We would like to thank public contributors who provided input and comment on earlier versions of this work. We would also like to thank the authors of the previous guidelines for reporting of embedded recruitment trials for their advice and input in the early stages of guideline development and to the independent reviewers and trialists who offered comments and piloting of the draft guideline. Funding Development of the SWAT reporting guideline was initiated as part of The PROMoting THE Use of SWATs (PROMETHEUS) programme, funded by the Medical Research Council (MRC) [grant number MR/R013748/1].Peer reviewe

Knee Replacement Bandaging Study (KReBS) Evaluating the effect of a Two-layer Compression Bandage System on Knee Function following Total Knee Arthroplasty: Study Protocol for a Randomised Controlled Trial

Background Data from a feasibility study suggests that the use of an inelastic, short-stretch compression bandage following total knee arthroplasty is a safe technique that may improve patient reported health outcomes, and that it is feasible to recruit to a full-scale study. Methods We will conduct a randomised controlled trial (RCT) of 2600 adult patients, which has 80% power to detect a 1 point difference in the Oxford Knee Score (a patient self-reported assessment of knee pain and function) at 52 weeks. Short stretch compression bandaging will be compared with standard wool and crepe bandaging following total knee arthroplasty. Recruitment will take place in orthopaedic units across the United Kingdom. Secondary outcomes include the EQ-5D-5L and EQ-5D-3L scores, pain, length of hospital stay and complications. Discussion The KReBS trial is a large study which aims to contribute to the evidence-base for informing clinical decisions for the use of compression bandaging following knee arthroplasty. Trial registration The trial is registered with the International Standard Randomised Controlled Trial Register (ISRCTN 87127065). Date of registration was 20/02/2017 (http://www.isrctn.com/ISRCTN87127065). Keywords Knee replacement; arthroplasty; compression bandage; randomised controlled tria

Bah humbug! Association between sending Christmas cards to trial participants and trial retention: randomised study within a trial conducted simultaneously across eight host trials

Objective To determine the effectiveness of sending Christmas cards to participants in randomised controlled trials to increase retention rate at follow-ups, and to explore the feasibility of doing a Study Within A Trial (SWAT) across multiple host trials simultaneously. Design Randomised SWAT conducted simultaneously across eight host trials. Setting Eight randomised controlled trials researching various areas including surgery and smoking cessation. Participants 3,223 participants who were still due at least one follow-up from their host randomised controlled trial. Intervention Participants were randomised (1:1, separately by each host trial) to either received a Christmas card in mid-December 2019 or to not receive a card. Main outcome measures Proportion of participants that completed their next follow-up (retention rate) within their host randomised controlled trial. Results 1469 participants (age 16-94 years; 70% (n=1033) female; 96% (813/847) white ethnicity) across the eight host randomised controlled trials were involved in the analysis (cut short owing to covid-19). No evidence was found of a difference in retention rate between the two arms for any of the host trials when analysed separately or when the results were combined (85.3% (639/749) for cards versus 85.4% (615/720) for no card; odds ratio 0.96, 95% confidence interval 0.71 to 1.29; P=0.77). No difference was observed when comparing just participants who were due a follow-up in the 30 days after receiving the card (odds ratio 0.96, 0.42 to 2.21). No evidence of a difference in time to complete the questionnaire was found (hazard ratio 1.01, 95% confidence interval 0.91 to 1.13; P=0.80). These results were robust to post hoc sensitivity analyses. The cost of this intervention was £0.76 (€0.91; $1.02) per participant, and it will have a carbon footprint of approximately 140 g CO2 equivalent per card. One benefit of this approach was the need to only submit one ethics application. Conclusions Sending Christmas cards to participants in randomised controlled trials does not increase retention. Undertaking a SWAT within multiple randomised controlled trials at the same time is, however, possible. This approach should be used more often to build an evidence base to support selection of recruitment and retention strategies. Although no evidence of a boost to retention was found, embedding a SWAT in multiple host trials simultaneously has been shown to be possible. Trial Registration SWAT 82: Sending Christmas cards to trial participants to improve retention (SWAT repository - https://www.qub.ac.uk/sites/TheNorthernIrelandNetworkforTrialsMethodologyResearch/FileStore/Filetoupload,846275,en.pdf#search=SWAT%2082