Indirect Comparisons: A Review of Reporting and Methodological Quality

Background: The indirect comparison of two interventions can be valuable in many situations. However, the quality of an indirect comparison will depend on several factors including the chosen methodology and validity of underlying assumptions. Published indirect comparisons are increasingly more common in the medical literature, but as yet, there are no published recommendations of how they should be reported. Our aim is to systematically review the quality of published indirect comparisons to add to existing empirical data suggesting that improvements can be made when reporting and applying indirect comparisons. Methodology/Findings: Reviews applying statistical methods to indirectly compare the clinical effectiveness of two interventions using randomised controlled trials were eligible. We searched (1966–2008) Database of Abstracts and Reviews of Effects, The Cochrane library, and Medline. Full review publications were assessed for eligibility. Specific criteria to assess quality were developed and applied. Forty-three reviews were included. Adequate methodology was used to calculate the indirect comparison in 41 reviews. Nineteen reviews assessed the similarity assumption using sensitivity analysis, subgroup analysis, or meta-regression. Eleven reviews compared trial-level characteristics. Twenty-four reviews assessed statistical homogeneity. Twelve reviews investigated causes of heterogeneity. Seventeen reviews included direct and indirect evidence for the same comparison; six reviews assessed consistency. One review combined both evidence types. Twentyfive reviews urged caution in interpretation of results, and 24 reviews indicated when results were from indirect evidence by stating this term with the result. Conclusions: This review shows that the underlying assumptions are not routinely explored or reported when undertaking indirect comparisons. We recommend, therefore, that the quality of indirect comparisons should be improved, in particular, by assessing assumptions and reporting the assessment methods applied. We propose that the quality criteria applied in this article may provide a basis to help review authors carry out indirect comparisons and to aid appropriate interpretation

Protocol for a systematic review of prognostic models for recurrent events in chronic conditions.

Background:Prognostic models for repeated events of the same type are highly useful in predicting when a patient may have a recurrence of a chronic disease or illness. Whilst methods are currently available for analysing recurrent event data in prognostic models, to our knowledge, most are not widely known or applied in a medical setting. As a result, often only the first recurrence is analysed meaning valuable information for multiple recurrences is discarded. Therefore, the aim of this review is to systemically review models for repeated medical events of the same type, to determine what modelling techniques are available and how they are applied. Methods:MEDLINE will be used as the primary method to search sources. Various databases from the Cochrane Library and EMBASE will also be searched. Trial registries such as Clinicaltrials.gov.uk will be searched, as will registered trials that are ongoing and not yet published. Abstracts submitted to conferences will also be searched, and non-English sources will also be considered. Studies to be included in the review will be decided based on PICO guidelines, where the study population and outcomes correspond to this study's aims and target population. The prognostic models used in each study chosen for inclusion in the review will be summarised qualitatively. Discussion:As recurrent event data is not widely analysed in prognostic models, the results from this systematic review will identify which methods are available and which are commonly used. It is also unknown if certain methods which will be identified in the review perform better given certain conditions. Therefore, if included studies assess predictive performance, the results of this review could also provide evidence to determine if certain models are better fitting dependant on the event rate of the chronic condition. The results will be used to determine if model selection varies across disease area. The review will also provide an insight into the development of any new methods used for analysing recurrent events. Trial registration:The review has been registered on PROSPERO (CRD42019116031)

Exploring treatment by covariate interactions using subgroup analysis and meta-regression in cochrane reviews:a review of recent practice

Treatment by covariate interactions can be explored in reviews using interaction analyses (e.g., subgroup analysis). Such analyses can provide information on how the covariate modifies the treatment effect and is an important methodological approach for personalising medicine. Guidance exists regarding how to apply such analyses but little is known about whether authors follow the guidance.Using published recommendations, we developed criteria to assess how well interaction analyses were designed, applied, interpreted, and reported. The Cochrane Database of Systematic Reviews was searched (8th August 2013). We applied the criteria to the most recently published review, with an accessible protocol, for each Cochrane Review Group. We excluded review updates, diagnostic test accuracy reviews, withdrawn reviews, and overviews of reviews. Data were summarised regarding reviews, covariates, and analyses.Each of the 52 included reviews planned or did interaction analyses; 51 reviews (98%) planned analyses and 33 reviews (63%) applied analyses. The type of analysis planned and the type subsequently applied (e.g., sensitivity or subgroup analysis) was discrepant in 24 reviews (46%). No review reported how or why each covariate had been chosen; 22 reviews (42%) did state each covariate a priori in the protocol but no review identified each post-hoc covariate as such. Eleven reviews (21%) mentioned five covariates or less. One review reported planning to use a method to detect interactions (i.e., interaction test) for each covariate; another review reported applying the method for each covariate. Regarding interpretation, only one review reported whether an interaction was detected for each covariate and no review discussed the importance, or plausibility, of the results, or the possibility of confounding for each covariate.Interaction analyses in Cochrane Reviews can be substantially improved. The proposed criteria can be used to help guide the reporting and conduct of analyses

The CONSENSUS study: protocol for a mixed methods study to establish which outcomes should be included in a core outcome set for oropharyngeal cancer

BACKGROUND: The incidence of oropharyngeal cancer is increasing in the developed world. This has led to a large rise in research activity and clinical trials in this area, yet there is no consensus on which outcomes should be measured. As a result, the outcomes measured often differ between trials of comparable interventions, making the combination or comparison of results between trials impossible. Outcomes may also be ‘cherry-picked’, such that favourable results are reported, and less favourable results withheld. The development of a minimum outcome reporting standard, known as a core outcome set, goes some way to addressing these problems. Core outcome sets are ideally developed using a patient-centred approach so that the outcomes measured are relevant to patients and clinical practice. Core outcome sets drive up the quality and relevance of research by ensuring that the right outcomes are consistently measured and reported in trials in specific areas of health or healthcare. METHODS/DESIGN: This is a mixed methods study involving three phases to develop a core outcome set for oropharyngeal cancer clinical trials. Firstly, a systematic review will establish which outcomes are measured in published oropharyngeal cancer randomised controlled trials (RCTs). Secondly, qualitative interviews with patients and carers in the UK and the USA will aim to establish which outcomes are important to these stakeholders. Data from these first two stages will be used to develop a comprehensive list of outcomes to be considered for inclusion in the core outcome set. In the third stage, patients and clinicians will participate in an iterative consensus exercise known as a Delphi study to refine the contents of the core outcome set. This protocol lays out the methodology to be implemented in the CONSENSUS study. DISCUSSION: A core outcome set defines a minimum outcome reporting standard for clinical trials in a particular area of health or healthcare. Its consistent implementation in oropharyngeal cancer clinical trials will improve the quality and relevance of research. TRIALS AND REGISTRATION: This study is registered at the National Institute for Health Research (NIHR) Clinical Research Network (CRN) portfolio, ID 13823 (17 January 2013)

Practical methods to pool multi-study joint longitudinal and time to event data

Background: Joint longitudinal and time-to-event data models have been established in a single study case as beneficial compared to separate longitudinal or time-to-event analyses in a range of cases, including data with study dropout, time-to-event models with longitudinal covariates measured with error, or cases when the relationship between longitudinal and time-to-event outcomes is of interest. However the methodology available for multi-study cases such as meta-analyses is limited. Aims: To investigate different approaches of modelling of multi-study joint longitudinal and time-to-event outcome data. Methods: Several methods are examined to account for between study heterogeneity, including as one stage methods that can include random effects at the study level, stratification of baseline hazard by study and use of fixed study indicator terms and their interactions with treatment assignment, or approaches for two stage pooling of joint model fits. These methods are applied to a real data example and further investigated in a simulation study. Software have been developed in R to allow these methods to be easily applied in future investigations, which will be available in a package alongside joineR collaboration. Results: The results from the real data example and simulation study will be presented

Correction to: joint models for longitudinal and time-to-event data: a review of reporting quality with a view to meta-analysis

Abstract Following publication of the original article [1] the authors reported that reference 15 (Cella et al.) had been incorrectly replaced with a duplicate of Brombin et al. during publication

Carbamazepine versus phenobarbitone monotherapy for epilepsy: an individual participant data review.

BACKGROUND:This is an updated version of the Cochrane Review previously published in 2016. This review is one in a series of Cochrane Reviews investigating pair-wise monotherapy comparisons.Epilepsy is a common neurological condition in which abnormal electrical discharges from the brain cause recurrent unprovoked seizures. It is believed that with effective drug treatment, up to 70% of individuals with active epilepsy have the potential to become seizure-free and go into long-term remission shortly after starting drug therapy with a single antiepileptic drug in monotherapy.Worldwide, carbamazepine and phenobarbitone are commonly used broad-spectrum antiepileptic drugs, suitable for most epileptic seizure types. Carbamazepine is a current first-line treatment for focal onset seizures, and is used in the USA and Europe. Phenobarbitone is no longer considered a first-line treatment because of concerns over associated adverse events, particularly documented behavioural adverse events in children treated with the drug. However, phenobarbitone is still commonly used in low- and middle-income countries because of its low cost. No consistent differences in efficacy have been found between carbamazepine and phenobarbitone in individual trials; however, the confidence intervals generated by these trials are wide, and therefore, synthesising the data of the individual trials may show differences in efficacy. OBJECTIVES:To review the time to treatment failure, remission and first seizure with carbamazepine compared with phenobarbitone when used as monotherapy in people with focal onset seizures (simple or complex focal and secondarily generalised), or generalised onset tonic-clonic seizures (with or without other generalised seizure types). SEARCH METHODS:For the latest update, we searched the following databases on 24 May 2018: the Cochrane Register of Studies (CRS Web), which includes Cochrane Epilepsy's Specialized Register and CENTRAL; MEDLINE; the US National Institutes of Health Ongoing Trials Register (ClinicalTrials.gov); and the World Health Organization International Clinical Trials Registry Platform (ICTRP). We handsearched relevant journals and contacted pharmaceutical companies, original trial investigators, and experts in the field. SELECTION CRITERIA:Randomised controlled trials comparing monotherapy with either carbamazepine or phenobarbitone in children or adults with focal onset seizures or generalised onset tonic-clonic seizures. DATA COLLECTION AND ANALYSIS:This was an individual participant data (IPD), review. Our primary outcome was time to treatment failure. Our secondary outcomes were time to first seizure post-randomisation, time to six-month remission, time to 12-month remission, and incidence of adverse events. We used Cox proportional hazards regression models to obtain trial-specific estimates of hazard ratios (HRs), with 95% confidence intervals (CIs), using the generic inverse variance method to obtain the overall pooled HR and 95% CI. MAIN RESULTS:We included 13 trials in this review and IPD were available for 836 individuals out of 1455 eligible individuals from six trials, 57% of the potential data. For remission outcomes, a HR of less than 1 indicates an advantage for phenobarbitone and for first seizure and treatment failure outcomes a HR of less than 1 indicates an advantage for carbamazepine.Results for the primary outcome of the review were: time to treatment failure for any reason related to treatment (pooled HR adjusted for seizure type for 676 participants: 0.66, 95% CI 0.50 to 0.86, moderate-quality evidence), time to treatment failure due to adverse events (pooled HR adjusted for seizure type for 619 participants: 0.69, 95% CI 0.49 to 0.97, low-quality evidence), time to treatment failure due to lack of efficacy (pooled HR adjusted for seizure type for 487 participants: 0.54, 95% CI 0.38 to 0.78, moderate-quality evidence), showing a statistically significant advantage for carbamazepine compared to phenobarbitone.For our secondary outcomes, we did not find any statistically significant differences between carbamazepine and phenobarbitone: time to first seizure post-randomisation (pooled HR adjusted for seizure type for 822 participants: 1.13, 95% CI 0.93 to 1.38, moderate-quality evidence), time to 12-month remission (pooled HR adjusted for seizure type for 683 participants: 1.09, 95% CI 0.84 to 1.40, low-quality evidence), and time to six-month remission pooled HR adjusted for seizure type for 683 participants: 1.01, 95% CI 0.81 to 1.24, low-quality evidence).Results of these secondary outcomes suggest that there may be an association between treatment effect in terms of efficacy and seizure type; that is, that participants with focal onset seizures experience seizure recurrence later and hence remission of seizures earlier on phenobarbitone than carbamazepine, and vice versa for individuals with generalised seizures. It is likely that the analyses of these outcomes were confounded by several methodological issues and misclassification of seizure type, which could have introduced the heterogeneity and bias into the results of this review.Limited information was available regarding adverse events in the trials and we could not compare the rates of adverse events between carbamazepine and phenobarbitone. Some adverse events reported on both drugs were abdominal pain, nausea, and vomiting, drowsiness, motor and cognitive disturbances, dysmorphic side effects (such as rash), and behavioural side effects in three paediatric trials. AUTHORS' CONCLUSIONS:Moderate-quality evidence from this review suggests that carbamazepine is likely to be a more effective drug than phenobarbitone in terms of treatment retention (treatment failures due to lack of efficacy or adverse events or both). Moderate- to low-quality evidence from this review also suggests an association between treatment efficacy and seizure type in terms of seizure recurrence and seizure remission, with an advantage for phenobarbitone for focal onset seizures and an advantage for carbamazepine for generalised onset seizures.However, some of the trials contributing to the analyses had methodological inadequacies and inconsistencies that may have impacted upon the results of this review. Therefore, we do not suggest that results of this review alone should form the basis of a treatment choice for a patient with newly onset seizures. We recommend that future trials should be designed to the highest quality possible with consideration of masking, choice of population, classification of seizure type, duration of follow-up, choice of outcomes and analysis, and presentation of results