Adjudicating outcomes in stroke trials

Central adjudication in randomised trials refers to the evaluation of outcome data by independent experts who are typically part of an event or outcome adjudication committee. Alternatively, local site investigators can assess outcomes, but these evaluations are often discarded in favour of the centrally adjudicated outcome. Central adjudication is thought to improve the precision of treatment effect estimates by reducing random error (non-differential misclassification), and in open-label studies adjudication has the potential to limit systematic error (differential misclassification) as adjudicators can always be blinded to treatment allocation. However, secondary analysis of trial data suggests that adjudication makes no meaningful difference to the endpoints of stroke and functional outcome. In addition, adjudication is a time-consuming and often expensive process. Given that central adjudication is common in trials investigating stroke, research is needed to establish what the benefits and costs are of adjudicating outcomes in stroke trials. In a systematic review, 15 randomised stroke trials (69,560 participants) were identified that had their primary outcome assessed by both central adjudicators and site investigators. The primary outcomes included were stroke (8 out of 15 trials, i.e. 8/15, 53%), a composite including stroke (6/15, 40%) and functional outcome after stroke (1/15, 7%). Overall, there was no evidence of any difference in treatment effect estimates based on data from central adjudicators and site investigators (pooled Ratio of Treatment Effects=1.02, 95% C.I:[0.95, 1.09]). This thesis also investigated whether two different approaches for contacting authors had any impact on the probability of receiving a response when trying to acquire data for a systematic review in a methodological trial. This nested randomised trial found that a short email with a protocol attached elicited a similar response compared to a longer email without a protocol. To explore circumstances where central adjudication would change the treatment effect estimate, five of the trials identified in the review were included in a further simulation study. Differential misclassification was introduced into each study via simulation until the treatment effect estimate was altered. For trials with a binary outcome, between 2.1% and 6% of participants needed to be differentially misclassified before this situation ensued. In addition, hypothetical trials were simulated with a binary outcome and varying sample size (1000-10000), overall event rate (10-50%), and treatment effect (0.67-0.90). Non-differential misclassification was introduced until the treatment effect was non-significant at 5% level. For these hypothetical trials, extensive non-differential misclassification was needed before the treatment effect became non-significant; trials with an overall event rate close to 50% and a larger sample size needed the highest proportion of random error before this occurred. Nine of the trials included in the review provided data on the cost of adjudication. These costs included adjudicators’ time, direct payments to adjudicators and co-ordinating centre costs. The number of events corrected after adjudication was the measure of benefit used. The mean cost per event corrected by adjudication was £2295.10. To investigate whether these findings were similar for adjudication of safety data, a case study was carried out using data from the Efficacy of Nitric Oxide in Stroke Trial. Serious adverse events were reported by site investigators who were not blinded to treatment allocation. Central blinded adjudicators reviewed the investigators’ report and used evidence available to confirm or re-categorise the classification of event. Repeating the main trial safety analysis with investigator reported events showed that adjudication had no effect on the main trial safety conclusions. To conclude, these studies have shown that central adjudication of the primary outcome in stroke trials does not alter treatment effect estimates. However, for studies without adequate blinding, a small amount of systematic error has the potential to alter the primary analysis and in this circumstance, adjudication is important. Given that the cost of central adjudication is not trivial, the potential advantages of adjudication may not outweigh cost and time disadvantages in stroke trials with blinded outcome assessment

Short email with attachment versus long email without attachment when contacting authors to request unpublished data for a systematic review: a nested randomised trial

Objective Systematic reviews often rely on the acquisition of unpublished analyses or data. We carried out a nested randomised trial comparing two different approaches for contacting authors to request additional data for a systematic review. Participants Participants were authors of published reports of prevention or treatment trials in stroke in which there was central adjudication of events. A primary and secondary research active author were selected as contacts for each trial. Interventions Authors were randomised to be sent either a short email with a protocol of the systematic review attached (‘Short’) or a longer email that contained detailed information and without the protocol attached (‘Long’). A maximum of two emails were sent to each author to obtain a response. The unit of analysis was trial, accounting for clustering by author. Primary and secondary outcome measures The primary outcome was whether a response was received from authors. Secondary outcomes included time to response, number of reminders needed before a response was received and whether authors agreed to collaborate. Results 88 trials with 76 primary authors were identified in the systematic review, and of these, 36 authors were randomised to Short (trials=45) and 40 to Long (trials=43). Responses were received for 69 trials. There was no evidence of a difference in response rate between trial arms (Short vs Long, OR 1.10, 95%CI 0.36 to 3.33). There was no evidence of a difference in time to response between trial arms (Short vs Long, HR 0.91, 95%CI 0.55 to 1.51). In total, 27% of authors responded within a day and 22% of authors never responded. Conclusions There was no evidence to suggest that email format had an impact on the number of responses received when acquiring data for a systematic review involving stroke trials or the time taken to receive these responses

The reporting standards of randomised controlled trials in leading medical journals between 2019 and 2020: a systematic review

Randomised controlled trials (RCTs) are the gold standard study design used to evaluate the safety and effectiveness of healthcare interventions. The reporting quality of RCTs is of fundamental importance for readers to appropriately analyse and understand the design and results of studies which are often labelled as practice changing papers. The aim of this article is to assess the reporting standards of a representative sample of randomised controlled trials (RCTs) published between 2019 and 2020 in four of the highest impact factor general medical journals. A systematic review of the electronic database Medline was conducted. Eligible RCTs included those published in the New England Journal of Medicine, Lancet, Journal of the American Medical Association, and British Medical Journal between January 1, 2019, and June 9, 2020. The study protocol was registered on medRxiv (https://doi.org/10.1101/2020.07.06.20147074). Of a total eligible sample of 498 studies, 50 full-text RCTs were reviewed against the CONSORT 2010 statement and relevant extensions where necessary. The mean adherence to the CONSORT checklist was 90% (SD 9%). There were specific items on the CONSORT checklist which had recurring suboptimal adherence, including in title (item 1a, 70% adherence), randomisation (items 9 and 10, 56% and 30% adherence) and outcomes and estimation (item 17b, 62% adherence). Amongst a sample of RCTs published in four of the highest impact factor general medical journals, there was good overall adherence to the CONSORT 2010 statement. However there remains significant room for improvement in areas such as description of allocation concealment and implementation of randomisation

Do investigator meetings improve recruitment rates in clinical trials? A retrospective before-and-after study of data from nine multi-centre clinical trials

© 2020 The Author(s). Background: Poor recruitment in clinical trials is well-documented. In large, multi-centre trials, communication between the coordinating centre and trial sites is essential. A commonly used communication tool is the hosting of an investigator/collaborator meeting, which offers an opportunity for sites to re-train and receive trial updates, learn from each other, share best practice and troubleshoot issues. Anecdotally, there is a perception that recruitment rates may increase after holding such a meeting. The aim of this before-and-after study was to examine any changes in recruitment after an investigator meeting. Methods: We conducted a retrospective study of nine trials at the Nottingham Clinical Trials Unit (NCTU) that were open to recruitment between 2014 and 2018. In the 8 weeks prior to the date of the investigator meeting, 82 sites (across nine trials) were open to recruitment; 60 of which attended the meeting, 22 who did not. Using meeting attendance data available in Trial Master Files (TMF) and recruitment data from randomisation datasets, we examined recruitment rates in the 8 weeks prior to and following the date of the investigator meeting. Results: For the 82 sites included, 284 participants were recruited in the 8 weeks prior to the meeting, with a further 300 participants recruited in the 8 weeks post meeting. This gives a mean change in weekly recruitment of 0.073 (- 0.129, 0.275) per site, demonstrating no statistically significant increase in recruitment after the investigator meeting. For the 60 attending sites, recruitment increased from 254 participants prior to the meeting to 271 post meeting, giving a 0.100 (- 0.160, 0.360) mean change in weekly recruitment per site, providing no evidence that recruitment rates increase following an investigator meeting. Conclusion: There is no statistical evidence to conclude that holding an investigator meeting increases recruitment in the 8 weeks following the meeting. Thus, if the meeting has been held in the belief that it will have a positive impact upon recruitment, trialists may wish to consider other evidence-based strategies known to increase recruitment rates. However, since there are a variety of reasons why an investigator meeting may be held, trialists should continue to consider this as a communication strategy with sites

Should we adjudicate outcomes in stroke trials? A systematic review

BACKGROUND: Central adjudication of outcomes is common in randomized clinical trials in stroke. The rationale for adjudication is clear; centrally adjudicated outcomes should have less random and systematic errors than outcomes assessed locally by site investigators. However, adjudication brings added complexities to a clinical trial and can be costly. AIM: To assess the evidence for outcome adjudication in stroke trials. SUMMARY OF REVIEW: We identified 12 studies evaluating central adjudication in stroke trials. The majority of these were secondary analyses of trials, and the results of all of these would have remained unchanged had central adjudication not taken place, even for trials without sufficient blinding. The largest differences between site-assessed and adjudicator-assessed outcomes were between the most subjective outcomes, such as causality of serious adverse events. We found that the cost of adjudication could be upward of £100,000 for medium to large prevention trials. These findings suggest that the cost of central adjudication may outweigh the advantages it brings in many cases. However, through simulation, we found that only a small amount of bias is required in site investigators’ outcome assessments before adjudication becomes important. CONCLUSION: Central adjudication may not be necessary in stroke trials with blinded outcome assessment. However, for open-label studies, central adjudication may be more important