Adjudicating outcomes in stroke trials

Abstract

Central adjudication in randomised trials refers to the evaluation of outcome data by independent experts who are typically part of an event or outcome adjudication committee. Alternatively, local site investigators can assess outcomes, but these evaluations are often discarded in favour of the centrally adjudicated outcome. Central adjudication is thought to improve the precision of treatment effect estimates by reducing random error (non-differential misclassification), and in open-label studies adjudication has the potential to limit systematic error (differential misclassification) as adjudicators can always be blinded to treatment allocation. However, secondary analysis of trial data suggests that adjudication makes no meaningful difference to the endpoints of stroke and functional outcome. In addition, adjudication is a time-consuming and often expensive process. Given that central adjudication is common in trials investigating stroke, research is needed to establish what the benefits and costs are of adjudicating outcomes in stroke trials. In a systematic review, 15 randomised stroke trials (69,560 participants) were identified that had their primary outcome assessed by both central adjudicators and site investigators. The primary outcomes included were stroke (8 out of 15 trials, i.e. 8/15, 53%), a composite including stroke (6/15, 40%) and functional outcome after stroke (1/15, 7%). Overall, there was no evidence of any difference in treatment effect estimates based on data from central adjudicators and site investigators (pooled Ratio of Treatment Effects=1.02, 95% C.I:[0.95, 1.09]). This thesis also investigated whether two different approaches for contacting authors had any impact on the probability of receiving a response when trying to acquire data for a systematic review in a methodological trial. This nested randomised trial found that a short email with a protocol attached elicited a similar response compared to a longer email without a protocol. To explore circumstances where central adjudication would change the treatment effect estimate, five of the trials identified in the review were included in a further simulation study. Differential misclassification was introduced into each study via simulation until the treatment effect estimate was altered. For trials with a binary outcome, between 2.1% and 6% of participants needed to be differentially misclassified before this situation ensued. In addition, hypothetical trials were simulated with a binary outcome and varying sample size (1000-10000), overall event rate (10-50%), and treatment effect (0.67-0.90). Non-differential misclassification was introduced until the treatment effect was non-significant at 5% level. For these hypothetical trials, extensive non-differential misclassification was needed before the treatment effect became non-significant; trials with an overall event rate close to 50% and a larger sample size needed the highest proportion of random error before this occurred. Nine of the trials included in the review provided data on the cost of adjudication. These costs included adjudicators’ time, direct payments to adjudicators and co-ordinating centre costs. The number of events corrected after adjudication was the measure of benefit used. The mean cost per event corrected by adjudication was £2295.10. To investigate whether these findings were similar for adjudication of safety data, a case study was carried out using data from the Efficacy of Nitric Oxide in Stroke Trial. Serious adverse events were reported by site investigators who were not blinded to treatment allocation. Central blinded adjudicators reviewed the investigators’ report and used evidence available to confirm or re-categorise the classification of event. Repeating the main trial safety analysis with investigator reported events showed that adjudication had no effect on the main trial safety conclusions. To conclude, these studies have shown that central adjudication of the primary outcome in stroke trials does not alter treatment effect estimates. However, for studies without adequate blinding, a small amount of systematic error has the potential to alter the primary analysis and in this circumstance, adjudication is important. Given that the cost of central adjudication is not trivial, the potential advantages of adjudication may not outweigh cost and time disadvantages in stroke trials with blinded outcome assessment