Adjudicating outcomes in stroke trials

Central adjudication in randomised trials refers to the evaluation of outcome data by independent experts who are typically part of an event or outcome adjudication committee. Alternatively, local site investigators can assess outcomes, but these evaluations are often discarded in favour of the centrally adjudicated outcome. Central adjudication is thought to improve the precision of treatment effect estimates by reducing random error (non-differential misclassification), and in open-label studies adjudication has the potential to limit systematic error (differential misclassification) as adjudicators can always be blinded to treatment allocation. However, secondary analysis of trial data suggests that adjudication makes no meaningful difference to the endpoints of stroke and functional outcome. In addition, adjudication is a time-consuming and often expensive process. Given that central adjudication is common in trials investigating stroke, research is needed to establish what the benefits and costs are of adjudicating outcomes in stroke trials. In a systematic review, 15 randomised stroke trials (69,560 participants) were identified that had their primary outcome assessed by both central adjudicators and site investigators. The primary outcomes included were stroke (8 out of 15 trials, i.e. 8/15, 53%), a composite including stroke (6/15, 40%) and functional outcome after stroke (1/15, 7%). Overall, there was no evidence of any difference in treatment effect estimates based on data from central adjudicators and site investigators (pooled Ratio of Treatment Effects=1.02, 95% C.I:[0.95, 1.09]). This thesis also investigated whether two different approaches for contacting authors had any impact on the probability of receiving a response when trying to acquire data for a systematic review in a methodological trial. This nested randomised trial found that a short email with a protocol attached elicited a similar response compared to a longer email without a protocol. To explore circumstances where central adjudication would change the treatment effect estimate, five of the trials identified in the review were included in a further simulation study. Differential misclassification was introduced into each study via simulation until the treatment effect estimate was altered. For trials with a binary outcome, between 2.1% and 6% of participants needed to be differentially misclassified before this situation ensued. In addition, hypothetical trials were simulated with a binary outcome and varying sample size (1000-10000), overall event rate (10-50%), and treatment effect (0.67-0.90). Non-differential misclassification was introduced until the treatment effect was non-significant at 5% level. For these hypothetical trials, extensive non-differential misclassification was needed before the treatment effect became non-significant; trials with an overall event rate close to 50% and a larger sample size needed the highest proportion of random error before this occurred. Nine of the trials included in the review provided data on the cost of adjudication. These costs included adjudicators’ time, direct payments to adjudicators and co-ordinating centre costs. The number of events corrected after adjudication was the measure of benefit used. The mean cost per event corrected by adjudication was £2295.10. To investigate whether these findings were similar for adjudication of safety data, a case study was carried out using data from the Efficacy of Nitric Oxide in Stroke Trial. Serious adverse events were reported by site investigators who were not blinded to treatment allocation. Central blinded adjudicators reviewed the investigators’ report and used evidence available to confirm or re-categorise the classification of event. Repeating the main trial safety analysis with investigator reported events showed that adjudication had no effect on the main trial safety conclusions. To conclude, these studies have shown that central adjudication of the primary outcome in stroke trials does not alter treatment effect estimates. However, for studies without adequate blinding, a small amount of systematic error has the potential to alter the primary analysis and in this circumstance, adjudication is important. Given that the cost of central adjudication is not trivial, the potential advantages of adjudication may not outweigh cost and time disadvantages in stroke trials with blinded outcome assessment

Estimating interactions and subgroup-specific treatment effects in meta-analysis without aggregation bias: A within-trial framework

Estimation of within-trial interactions in meta-analysis is crucial for reliable assessment of how treatment effects vary across participant subgroups. However, current methods have various limitations. Patients, clinicians and policy-makers need reliable estimates of treatment effects within specific covariate subgroups, on relative and absolute scales, in order to target treatments appropriately - which estimation of an interaction effect does not in itself provide. Also, the focus has been on covariates with only two subgroups, and may exclude relevant data if only a single subgroup is reported. Therefore, in this article we further develop the "within-trial" framework by providing practical methods to (1) estimate within-trial interactions across two or more subgroups; (2) estimate subgroup-specific ("floating") treatment effects that are compatible with the within-trial interactions and make maximum use of available data; and (3) clearly present this data using novel implementation of forest plots. We described the steps involved and apply the methods to two examples taken from previously published meta-analyses, and demonstrate a straightforward implementation in Stata based upon existing code for multivariate meta-analysis. We discuss how the within-trial framework and plots can be utilised with aggregate (or "published") source data, as well as with individual participant data, to effectively demonstrate how treatment effects differ across participant subgroups

Short email with attachment versus long email without attachment when contacting authors to request unpublished data for a systematic review: a nested randomised trial

Objective: Systematic reviews often rely on the acquisition of unpublished analyses or data. We carried out a nested randomised trial comparing two different approaches for contacting authors to request additional data for a systematic review. Participants: Participants were authors of published reports of prevention or treatment trials in stroke in which there was central adjudication of events. A primary and secondary research active author were selected as contacts for each trial. Interventions: Authors were randomised to be sent either a short email with a protocol of the systematic review attached (“Short”), or a longer email that contained detailed information and without the protocol attached (“Long”). A maximum of two emails were sent to each author to obtain a response. The unit of analysis was trial, accounting for clustering by author.Primary and secondary outcome measures: The primary outcome was whether a response was received from authors. Secondary outcomes included time to response, number of reminders needed before a response was received and whether authors agreed to collaborate.Results: 88 trials with 76 primary authors were identified in the systematic review, and of these, 36 authors were randomised to Short [trials=45], and 40 to Long [trials=43]. Responses were received for 69 trials. There was no evidence of a difference in response rate between trial arms (Short vs Long, odds ratio 1.10, 95% C.I: [0.36, 3.33]). There was no evidence of a difference in time to response between trial arms (Short vs Long, hazard ratio 0.91, 95% C.I: [0.55, 1.51]). In total, 27% of authors responded within a day and 22% of authors never responded. Conclusions: There was no evidence to suggest that email format had an impact on the number of responses received when acquiring data for a systematic review involving stroke trials, or the time taken to receive these responses

Use of multiple covariates in assessing treatment-effect modifiers: A methodological review of individual participant data meta-analyses

Individual participant data (IPD) meta-analyses of randomised trials are considered a reliable way to assess participant-level treatment effect modifiers but may not make the best use of the available data. Traditionally, effect modifiers are explored one covariate at a time, which gives rise to the possibility that evidence of treatment-covariate interaction may be due to confounding from a different, related covariate. We aimed to evaluate current practice when estimating treatment-covariate interactions in IPD meta-analysis, specifically focusing on involvement of additional covariates in the models. We reviewed 100 IPD meta-analyses of randomised trials, published between 2015 and 2020, that assessed at least one treatment-covariate interaction. We identified four approaches to handling additional covariates: (1) Single interaction model (unadjusted): No additional covariates included (57/100 IPD meta-analyses); (2) Single interaction model (adjusted): Adjustment for the main effect of at least one additional covariate (35/100); (3) Multiple interactions model: Adjustment for at least one two-way interaction between treatment and an additional covariate (3/100); and (4) Three-way interaction model: Three-way interaction formed between treatment, the additional covariate and the potential effect modifier (5/100). IPD is not being utilised to its fullest extent. In an exemplar dataset, we demonstrate how these approaches lead to different conclusions. Researchers should adjust for additional covariates when estimating interactions in IPD meta-analysis providing they adjust their main effects, which is already widely recommended. Further, they should consider whether more complex approaches could provide better information on who might benefit most from treatments, improving patient choice and treatment policy and practice

Cost-benefit of outcome adjudication in nine randomised stroke trials

Background: Central adjudication of outcomes is common for randomised trials and should control for differential misclassification. However, few studies have estimated the cost of the adjudication process.Methods: We estimated the cost of adjudicating the primary outcome in nine randomised stroke trials (25,436 participants). The costs included adjudicators’ time, direct payments to adjudicators, and co-ordinating centre costs (e.g. uploading cranial scans and general set-up costs). The number of events corrected after adjudication was our measure of benefit. We calculated cost per corrected event for each trial and in total.Results: The primary outcome in all nine trials was either stroke or a composite that included stroke. In total, the adjudication process associated with this primary outcome cost in excess of £100,000 for a third of the trials (3/9). Mean cost per event corrected by adjudication was £2295.10 (standard deviation: £1482.42).Conclusions: Central adjudication is a time-consuming and potentially costly process. These costs need to be considered when designing a trial and should be evaluated alongside the potential benefits adjudication brings to determine whether they outweigh this expense

Standard care informed by the result of a placental growth factor blood test versus standard care alone in women with reduced fetal movement at or after 36+0 weeks’ gestation: a pilot randomised controlled trial

BackgroundBiomarkers of placental function can potentially aid the diagnosis and prediction of pregnancy complications. This randomised controlled pilot trial assessed whether for women with reduced fetal movement (RFM), intervention directed by the measurement of a placental biomarker in addition to standard care was feasible and improved pregnancy outcome compared with standard care alone.MethodsWomen aged 16–50 years presenting at eight UK maternity units with RFM between 36+0 and 41+0 weeks’ gestation with a viable singleton pregnancy and no indication for immediate delivery were eligible. Participants were randomised 1:1 in an unblinded manner to standard care and a biomarker blood test result revealed and acted on (intervention arm) or standard care where the biomarker result was not available (control arm). The objectives were to determine the feasibility of a main trial by recruiting 175–225 participants over 9 months and to provide proof of concept that informing care by measurement of placental biomarkers may improve outcome. Feasibility was assessed via the number of potentially eligible women, number recruited, reasons for non-recruitment and compliance. Proof of concept outcomes included the rates of the induction of labour and caesarean birth, and a composite adverse pregnancy outcome.ResultsOverall, 2917 women presented with RFM ≥ 36 weeks, 352 were approached to participate and 216 (61%) were randomised (intervention n = 109, control n = 107). The main reason for not approaching women was resource/staff issues (n = 1510). Ninety-seven women declined the trial, mainly due to not liking blood tests (n = 24) or not wanting to be in a trial (n = 21). Compliance with the trial interventions was 100% in both arms. Labour was induced in 97 (45%) participants (intervention n = 49, control n = 48), while 17 (9%) had planned caesarean sections (intervention n = 9, control n = 8). Overall, 9 (8%) babies in the intervention arm had the composite adverse pregnancy outcome versus 4 (4%) in the control arm.ConclusionsA main trial using a placental biomarker in combination with delivery, as indicated by the biomarker, in women with RFM is feasible. The frequency of adverse outcomes in this population is low, hence, a large sample size would be required along with consideration of the most appropriate outcome measures

Association between tocilizumab, sarilumab and all-cause mortality at 28 days in hospitalised patients with COVID-19:A network meta-analysis

BACKGROUND: A recent prospective meta-analysis demonstrated that interleukin-6 antagonists are associated with lower all-cause mortality in hospitalised patients with COVID-19, compared with usual care or placebo. However, emerging evidence suggests that clinicians are favouring the use of tocilizumab over sarilumab. A new randomised comparison of these agents from the REMAP-CAP trial shows similar effects on in-hospital mortality. Therefore, we initiated a network meta-analysis, to estimate pairwise associations between tocilizumab, sarilumab and usual care or placebo with 28-day mortality, in COVID-19 patients receiving concomitant corticosteroids and ventilation, based on all available direct and indirect evidence. METHODS: Eligible trials randomised hospitalised patients with COVID-19 that compared tocilizumab or sarilumab with usual care or placebo in the prospective meta-analysis or that directly compared tocilizumab with sarilumab. Data were restricted to patients receiving corticosteroids and either non-invasive or invasive ventilation at randomisation. Pairwise associations between tocilizumab, sarilumab and usual care or placebo for all-cause mortality 28 days after randomisation were estimated using a frequentist contrast-based network meta-analysis of odds ratios (ORs), implementing multivariate fixed-effects models that assume consistency between the direct and indirect evidence. FINDINGS: One trial (REMAP-CAP) was identified that directly compared tocilizumab with sarilumab and supplied results on all-cause mortality at 28-days. This network meta-analysis was based on 898 eligible patients (278 deaths) from REMAP-CAP and 3710 eligible patients from 18 trials (1278 deaths) from the prospective meta-analysis. Summary ORs were similar for tocilizumab [0·82 [0·71–0·95, p = 0·008]] and sarilumab [0·80 [0·61–1·04, p = 0·09]] compared with usual care or placebo. The summary OR for 28-day mortality comparing tocilizumab with sarilumab was 1·03 [95%CI 0·81–1·32, p = 0·80]. The p-value for the global test of inconsistency was 0·28. CONCLUSIONS: Administration of either tocilizumab or sarilumab was associated with lower 28-day all-cause mortality compared with usual care or placebo. The association is not dependent on the choice of interleukin-6 receptor antagonist

Which patients with metastatic hormone-sensitive prostate cancer benefit from docetaxel: a systematic review and meta-analysis of individual participant data from randomised trials

BACKGROUND: Adding docetaxel to androgen deprivation therapy (ADT) improves survival in patients with metastatic, hormone-sensitive prostate cancer, but uncertainty remains about who benefits most. We therefore aimed to obtain up-to-date estimates of the overall effects of docetaxel and to assess whether these effects varied according to prespecified characteristics of the patients or their tumours. METHODS: The STOPCAP M1 collaboration conducted a systematic review and meta-analysis of individual participant data. We searched MEDLINE (from database inception to March 31, 2022), Embase (from database inception to March 31, 2022), the Cochrane Central Register of Controlled Trials (from database inception to March 31, 2022), proceedings of relevant conferences (from Jan 1, 1990, to Dec 31, 2022), and ClinicalTrials.gov (from database inception to March 28, 2023) to identify eligible randomised trials that assessed docetaxel plus ADT compared with ADT alone in patients with metastatic, hormone-sensitive prostate cancer. Detailed and updated individual participant data were requested directly from study investigators or through relevant repositories. The primary outcome was overall survival. Secondary outcomes were progression-free survival and failure-free survival. Overall pooled effects were estimated using an adjusted, intention-to-treat, two-stage, fixed-effect meta-analysis, with one-stage and random-effects sensitivity analyses. Missing covariate values were imputed. Differences in effect by participant characteristics were estimated using adjusted two-stage, fixed-effect meta-analysis of within-trial interactions on the basis of progression-free survival to maximise power. Identified effect modifiers were also assessed on the basis of overall survival. To explore multiple subgroup interactions and derive subgroup-specific absolute treatment effects we used one-stage flexible parametric modelling and regression standardisation. We assessed the risk of bias using the Cochrane Risk of Bias 2 tool. This study is registered with PROSPERO, CRD42019140591. FINDINGS: We obtained individual participant data from 2261 patients (98% of those randomised) from three eligible trials (GETUG-AFU15, CHAARTED, and STAMPEDE trials), with a median follow-up of 72 months (IQR 55-85). Individual participant data were not obtained from two additional small trials. Based on all included trials and patients, there were clear benefits of docetaxel on overall survival (hazard ratio [HR] 0·79, 95% CI 0·70 to 0·88; p<0·0001), progression-free survival (0·70, 0·63 to 0·77; p<0·0001), and failure-free survival (0·64, 0·58 to 0·71; p<0·0001), representing 5-year absolute improvements of around 9-11%. The overall risk of bias was assessed to be low, and there was no strong evidence of differences in effect between trials for all three main outcomes. The relative effect of docetaxel on progression-free survival appeared to be greater with increasing clinical T stage (pinteraction=0·0019), higher volume of metastases (pinteraction=0·020), and, to a lesser extent, synchronous diagnosis of metastatic disease (pinteraction=0·077). Taking into account the other interactions, the effect of docetaxel was independently modified by volume and clinical T stage, but not timing. There was no strong evidence that docetaxel improved absolute effects at 5 years for patients with low-volume, metachronous disease (-1%, 95% CI -15 to 12, for progression-free survival; 0%, -10 to 12, for overall survival). The largest absolute improvement at 5 years was observed for those with high-volume, clinical T stage 4 disease (27%, 95% CI 17 to 37, for progression-free survival; 35%, 24 to 47, for overall survival). INTERPRETATION: The addition of docetaxel to hormone therapy is best suited to patients with poorer prognosis for metastatic, hormone-sensitive prostate cancer based on a high volume of disease and potentially the bulkiness of the primary tumour. There is no evidence of meaningful benefit for patients with metachronous, low-volume disease who should therefore be managed differently. These results will better characterise patients most and, importantly, least likely to gain benefit from docetaxel, potentially changing international practice, guiding clinical decision making, better informing treatment policy, and improving patient outcomes. FUNDING: UK Medical Research Council and Prostate Cancer UK

Prehospital transdermal glyceryl trinitrate in patients with ultra-acute presumed stroke (RIGHT-2): an ambulance-based, randomised, sham-controlled, blinded, phase 3 trial

Background: High blood pressure is common in acute stroke and is a predictor of poor outcome; however, large trials of lowering blood pressure have given variable results, and the management of high blood pressure in ultra-acute stroke remains unclear. We investigated whether transdermal glyceryl trinitrate (GTN; also known as nitroglycerin), a nitric oxide donor, might improve outcome when administered very early after stroke onset. Methods: We did a multicentre, paramedic-delivered, ambulance-based, prospective, randomised, sham-controlled, blinded-endpoint, phase 3 trial in adults with presumed stroke within 4 h of onset, face-arm-speech-time score of 2 or 3, and systolic blood pressure 120 mm Hg or higher. Participants were randomly assigned (1:1) to receive transdermal GTN (5 mg once daily for 4 days; the GTN group) or a similar sham dressing (the sham group) in UK-based ambulances by paramedics, with treatment continued in hospital. Paramedics were unmasked to treatment, whereas participants were masked. The primary outcome was the 7-level modified Rankin Scale (mRS; a measure of functional outcome) at 90 days, assessed by central telephone follow-up with masking to treatment. Analysis was hierarchical, first in participants with a confirmed stroke or transient ischaemic attack (cohort 1), and then in all participants who were randomly assigned (intention to treat, cohort 2) according to the statistical analysis plan. This trial is registered with ISRCTN, number ISRCTN26986053. Findings: Between Oct 22, 2015, and May 23, 2018, 516 paramedics from eight UK ambulance services recruited 1149 participants (n=568 in the GTN group, n=581 in the sham group). The median time to randomisation was 71 min (IQR 45–116). 597 (52%) patients had ischaemic stroke, 145 (13%) had intracerebral haemorrhage, 109 (9%) had transient ischaemic attack, and 297 (26%) had a non-stroke mimic at the final diagnosis of the index event. In the GTN group, participants' systolic blood pressure was lowered by 5·8 mm Hg compared with the sham group (p&lt;0·0001), and diastolic blood pressure was lowered by 2·6 mm Hg (p=0·0026) at hospital admission. We found no difference in mRS between the groups in participants with a final diagnosis of stroke or transient ischaemic stroke (cohort 1): 3 (IQR 2–5; n=420) in the GTN group versus 3 (2–5; n=408) in the sham group, adjusted common odds ratio for poor outcome 1·25 (95% CI 0·97–1·60; p=0·083); we also found no difference in mRS between all patients (cohort 2: 3 [2–5]; n=544, in the GTN group vs 3 [2–5]; n=558, in the sham group; 1·04 [0·84–1·29]; p=0·69). We found no difference in secondary outcomes, death (treatment-related deaths: 36 in the GTN group vs 23 in the sham group [p=0·091]), or serious adverse events (188 in the GTN group vs 170 in the sham group [p=0·16]) between treatment groups. Interpretation: Prehospital treatment with transdermal GTN does not seem to improve functional outcome in patients with presumed stroke. It is feasible for UK paramedics to obtain consent and treat patients with stroke in the ultra-acute prehospital setting

Investigating the effect of independent blinded digital image assessment on the STOP GAP trial

Background Blinding is the process of keeping treatment assignment hidden and is used to minimise the possibility of bias. Trials at high risk of bias have been shown to report larger treatment effects than low risk studies. In dermatology, one popular method of blinding is to have independent outcome assessors who are unaware of treatment allocation assessing the end point using digital photographs. However, this can be complex, expensive and time-consuming. The objective of this study was to compare the effect of blinded and unblinded outcome assessment on the results of the STOP GAP trial. Methods The STOP GAP trial compared prednisolone to ciclosporin in treating pyoderma gangrenosum. Participants’ lesions were measured at baseline and 6 weeks to calculate the primary outcome, speed of healing. Independent blinded assessors obtained measurements from digital photographs using specialist software. In addition, unblinded treating clinicians estimated lesion area by measuring length and width. The primary outcome was determined using blinded measurements where available, otherwise unblinded measurements were used (method referred to as trial measurements). In this study, agreement between the trial and unblinded measurements was determined using the intraclass correlation coefficient (ICC). The STOP GAP primary analysis was repeated using unblinded measurements only. We introduced differential and non-differential error in unblinded measurements and investigated the effect on the STOP GAP primary analysis. Results 86 (80%) of the 108 patients were assessed using digital images. Agreement between trial and unblinded measurements was excellent (ICC=0.92 at baseline; 0.83 at 6 weeks). There was no evidence that the results of the trial primary analysis differed according to how the primary outcome was assessed (p-value for homogeneity = 1.00). Conclusions Blinded digital image assessment in STOP GAP did not meaningfully alter trial conclusions compared with unblinded assessment. However, as the process brought added accuracy and credibility to the trial it was considered worthwhile. These findings question the usefulness of digital image assessment in a trial with an objective outcome and where bias is not expected to be excessive. Further research should investigate if there are alternative, less complex ways of incorporating blinding in clinical trials