Indirect Comparisons: A Review of Reporting and Methodological Quality

Background: The indirect comparison of two interventions can be valuable in many situations. However, the quality of an indirect comparison will depend on several factors including the chosen methodology and validity of underlying assumptions. Published indirect comparisons are increasingly more common in the medical literature, but as yet, there are no published recommendations of how they should be reported. Our aim is to systematically review the quality of published indirect comparisons to add to existing empirical data suggesting that improvements can be made when reporting and applying indirect comparisons. Methodology/Findings: Reviews applying statistical methods to indirectly compare the clinical effectiveness of two interventions using randomised controlled trials were eligible. We searched (1966–2008) Database of Abstracts and Reviews of Effects, The Cochrane library, and Medline. Full review publications were assessed for eligibility. Specific criteria to assess quality were developed and applied. Forty-three reviews were included. Adequate methodology was used to calculate the indirect comparison in 41 reviews. Nineteen reviews assessed the similarity assumption using sensitivity analysis, subgroup analysis, or meta-regression. Eleven reviews compared trial-level characteristics. Twenty-four reviews assessed statistical homogeneity. Twelve reviews investigated causes of heterogeneity. Seventeen reviews included direct and indirect evidence for the same comparison; six reviews assessed consistency. One review combined both evidence types. Twentyfive reviews urged caution in interpretation of results, and 24 reviews indicated when results were from indirect evidence by stating this term with the result. Conclusions: This review shows that the underlying assumptions are not routinely explored or reported when undertaking indirect comparisons. We recommend, therefore, that the quality of indirect comparisons should be improved, in particular, by assessing assumptions and reporting the assessment methods applied. We propose that the quality criteria applied in this article may provide a basis to help review authors carry out indirect comparisons and to aid appropriate interpretation

Protocol for a systematic review of prognostic models for recurrent events in chronic conditions.

Background:Prognostic models for repeated events of the same type are highly useful in predicting when a patient may have a recurrence of a chronic disease or illness. Whilst methods are currently available for analysing recurrent event data in prognostic models, to our knowledge, most are not widely known or applied in a medical setting. As a result, often only the first recurrence is analysed meaning valuable information for multiple recurrences is discarded. Therefore, the aim of this review is to systemically review models for repeated medical events of the same type, to determine what modelling techniques are available and how they are applied. Methods:MEDLINE will be used as the primary method to search sources. Various databases from the Cochrane Library and EMBASE will also be searched. Trial registries such as Clinicaltrials.gov.uk will be searched, as will registered trials that are ongoing and not yet published. Abstracts submitted to conferences will also be searched, and non-English sources will also be considered. Studies to be included in the review will be decided based on PICO guidelines, where the study population and outcomes correspond to this study's aims and target population. The prognostic models used in each study chosen for inclusion in the review will be summarised qualitatively. Discussion:As recurrent event data is not widely analysed in prognostic models, the results from this systematic review will identify which methods are available and which are commonly used. It is also unknown if certain methods which will be identified in the review perform better given certain conditions. Therefore, if included studies assess predictive performance, the results of this review could also provide evidence to determine if certain models are better fitting dependant on the event rate of the chronic condition. The results will be used to determine if model selection varies across disease area. The review will also provide an insight into the development of any new methods used for analysing recurrent events. Trial registration:The review has been registered on PROSPERO (CRD42019116031)

Exploring treatment by covariate interactions using subgroup analysis and meta-regression in cochrane reviews:a review of recent practice

Treatment by covariate interactions can be explored in reviews using interaction analyses (e.g., subgroup analysis). Such analyses can provide information on how the covariate modifies the treatment effect and is an important methodological approach for personalising medicine. Guidance exists regarding how to apply such analyses but little is known about whether authors follow the guidance.Using published recommendations, we developed criteria to assess how well interaction analyses were designed, applied, interpreted, and reported. The Cochrane Database of Systematic Reviews was searched (8th August 2013). We applied the criteria to the most recently published review, with an accessible protocol, for each Cochrane Review Group. We excluded review updates, diagnostic test accuracy reviews, withdrawn reviews, and overviews of reviews. Data were summarised regarding reviews, covariates, and analyses.Each of the 52 included reviews planned or did interaction analyses; 51 reviews (98%) planned analyses and 33 reviews (63%) applied analyses. The type of analysis planned and the type subsequently applied (e.g., sensitivity or subgroup analysis) was discrepant in 24 reviews (46%). No review reported how or why each covariate had been chosen; 22 reviews (42%) did state each covariate a priori in the protocol but no review identified each post-hoc covariate as such. Eleven reviews (21%) mentioned five covariates or less. One review reported planning to use a method to detect interactions (i.e., interaction test) for each covariate; another review reported applying the method for each covariate. Regarding interpretation, only one review reported whether an interaction was detected for each covariate and no review discussed the importance, or plausibility, of the results, or the possibility of confounding for each covariate.Interaction analyses in Cochrane Reviews can be substantially improved. The proposed criteria can be used to help guide the reporting and conduct of analyses

The CONSENSUS study: protocol for a mixed methods study to establish which outcomes should be included in a core outcome set for oropharyngeal cancer

BACKGROUND: The incidence of oropharyngeal cancer is increasing in the developed world. This has led to a large rise in research activity and clinical trials in this area, yet there is no consensus on which outcomes should be measured. As a result, the outcomes measured often differ between trials of comparable interventions, making the combination or comparison of results between trials impossible. Outcomes may also be ‘cherry-picked’, such that favourable results are reported, and less favourable results withheld. The development of a minimum outcome reporting standard, known as a core outcome set, goes some way to addressing these problems. Core outcome sets are ideally developed using a patient-centred approach so that the outcomes measured are relevant to patients and clinical practice. Core outcome sets drive up the quality and relevance of research by ensuring that the right outcomes are consistently measured and reported in trials in specific areas of health or healthcare. METHODS/DESIGN: This is a mixed methods study involving three phases to develop a core outcome set for oropharyngeal cancer clinical trials. Firstly, a systematic review will establish which outcomes are measured in published oropharyngeal cancer randomised controlled trials (RCTs). Secondly, qualitative interviews with patients and carers in the UK and the USA will aim to establish which outcomes are important to these stakeholders. Data from these first two stages will be used to develop a comprehensive list of outcomes to be considered for inclusion in the core outcome set. In the third stage, patients and clinicians will participate in an iterative consensus exercise known as a Delphi study to refine the contents of the core outcome set. This protocol lays out the methodology to be implemented in the CONSENSUS study. DISCUSSION: A core outcome set defines a minimum outcome reporting standard for clinical trials in a particular area of health or healthcare. Its consistent implementation in oropharyngeal cancer clinical trials will improve the quality and relevance of research. TRIALS AND REGISTRATION: This study is registered at the National Institute for Health Research (NIHR) Clinical Research Network (CRN) portfolio, ID 13823 (17 January 2013)

Practical methods to pool multi-study joint longitudinal and time to event data

Background: Joint longitudinal and time-to-event data models have been established in a single study case as beneficial compared to separate longitudinal or time-to-event analyses in a range of cases, including data with study dropout, time-to-event models with longitudinal covariates measured with error, or cases when the relationship between longitudinal and time-to-event outcomes is of interest. However the methodology available for multi-study cases such as meta-analyses is limited. Aims: To investigate different approaches of modelling of multi-study joint longitudinal and time-to-event outcome data. Methods: Several methods are examined to account for between study heterogeneity, including as one stage methods that can include random effects at the study level, stratification of baseline hazard by study and use of fixed study indicator terms and their interactions with treatment assignment, or approaches for two stage pooling of joint model fits. These methods are applied to a real data example and further investigated in a simulation study. Software have been developed in R to allow these methods to be easily applied in future investigations, which will be available in a package alongside joineR collaboration. Results: The results from the real data example and simulation study will be presented

Correction to: joint models for longitudinal and time-to-event data: a review of reporting quality with a view to meta-analysis

Abstract Following publication of the original article [1] the authors reported that reference 15 (Cella et al.) had been incorrectly replaced with a duplicate of Brombin et al. during publication

Systematic review of methods used in prediction models with recurrent event data.

BackgroundPatients who suffer from chronic conditions or diseases are susceptible to experiencing repeated events of the same type (e.g. seizures), termed 'recurrent events'. Prediction models can be used to predict the risk of recurrence so that intervention or management can be tailored accordingly, but statistical methodology can vary. The objective of this systematic review was to identify and describe statistical approaches that have been applied for the development and validation of multivariable prediction models with recurrent event data. A secondary objective was to informally assess the characteristics and quality of analysis approaches used in the development and validation of prediction models of recurrent event data.MethodsSearches were run in MEDLINE using a search strategy in 2019 which included index terms and phrases related to recurrent events and prediction models. For studies to be included in the review they must have developed or validated a multivariable clinical prediction model for recurrent event outcome data, specifically modelling the recurrent events and the timing between them. The statistical analysis methods used to analyse the recurrent event data in the clinical prediction model were extracted to answer the primary aim of the systematic review. In addition, items such as the event rate as well as any discrimination and calibration statistics that were used to assess the model performance were extracted for the secondary aim of the review.ResultsA total of 855 publications were identified using the developed search strategy and 301 of these are included in our systematic review. The Andersen-Gill method was identified as the most commonly applied method in the analysis of recurrent events, which was used in 152 (50.5%) studies. This was closely followed by frailty models which were used in 116 (38.5%) included studies. Of the 301 included studies, only 75 (24.9%) internally validated their model(s) and three (1.0%) validated their model(s) in an external dataset.ConclusionsThis review identified a variety of methods which are used in practice when developing or validating prediction models for recurrent events. The variability of the approaches identified is cause for concern as it indicates possible immaturity in the field and highlights the need for more methodological research to bring greater consistency in approach of recurrent event analysis. Further work is required to ensure publications report all required information and use robust statistical methods for model development and validation.Prospero registrationCRD42019116031