Test-treatment RCTs are susceptible to bias:a review of the methodological quality of randomized trials that evaluate diagnostic tests

Abstract Background There is a growing recognition for the need to expand our evidence base for the clinical effectiveness of diagnostic tests. Many international bodies are calling for diagnostic randomized controlled trials to provide the most rigorous evidence of impact to patient health. Although these so-called test-treatment RCTs are very challenging to undertake due to their methodological complexity, they have not been subjected to a systematic appraisal of their methodological quality. The extent to which these trials may be producing biased results therefore remains unknown. We set out to address this issue by conducting a methodological review of published test-treatment trials to determine how often they implement adequate methods to limit bias and safeguard the validity of results. Methods We ascertained all test-treatment RCTs published 2004–2007, indexed in CENTRAL, including RCTs which randomized patients to diagnostic tests and measured patient outcomes after treatment. Tests used for screening, monitoring or prognosis were excluded. We assessed adequacy of sequence generation, allocation concealment and intention-to-treat, appropriateness of primary analyses, blinding and reporting of power calculations, and extracted study characteristics including the primary outcome. Results One hundred three trials compared 105 control with 119 experimental interventions, and reported 150 primary outcomes. Randomization and allocation concealment were adequate in 57 and 37% of trials. Blinding was uncommon (patients 5%, clinicians 4%, outcome assessors 21%), as was an adequate intention-to-treat analysis (29%). Overall 101 of 103 trials (98%) were at risk of bias, as judged using standard Cochrane criteria. Conclusion Test-treatment trials are particularly susceptible to attrition and inadequate primary analyses, lack of blinding and under-powering. These weaknesses pose much greater methodological and practical challenges to conducting reliable RCT evaluations of test-treatment strategies than standard treatment interventions. We suggest a cautious approach that first examines whether a test-treatment intervention can accommodate the methodological safeguards necessary to minimize bias, and highlight that test-treatment RCTs require different methods to ensure reliability than standard treatment trials. Please see the companion paper to this article: http://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-016-0286-0

Molecular networks involved in mouse cerebral corticogenesis and spatio-temporal regulation of Sox4 and Sox11 novel antisense transcripts revealed by transcriptome profiling.

Background: Development of the cerebral cortex requires highly specific spatio-temporal regulation of gene expression. It is proposed that transcriptome profiling of the cerebral cortex at various developmental time points or regions will reveal candidate genes and associated molecular pathways involved in cerebral corticogenesis. Results: Serial analysis of gene expression (SAGE) libraries were constructed from C57BL/6 mouse cerebral cortices of age embryonic day (E) 15.5, E17.5, postnatal day (P) 1.5 and 4 to 6 months. Hierarchical clustering analysis of 561 differentially expressed transcripts showed regionalized, stage-specific and co-regulated expression profiles. SAGE expression profiles of 70 differentially expressed transcripts were validated using quantitative RT-PCR assays. Ingenuity pathway analyses of validated differentially expressed transcripts demonstrated that these transcripts possess distinctive functional properties related to various stages of cerebral corticogenesis and human neurological disorders. Genomic clustering analysis of the differentially expressed transcripts identified two highly transcribed genomic loci, Sox4 and Sox11, during embryonic cerebral corticogenesis. These loci feature unusual overlapping sense and antisense transcripts with alternative polyadenylation sites and differential expression. The Sox4 and Sox11 antisense transcripts were highly expressed in the brain compared to other mouse organs and are differentially expressed in both the proliferating and differentiating neural stem/progenitor cells and P19 (embryonal carcinoma) cells. Conclusions: We report validated gene expression profiles that have implications for understanding the associations between differentially expressed transcripts, novel targets and related disorders pertaining to cerebral corticogenesis. The study reports, for the first time, spatio-temporally regulated Sox4 and Sox11 antisense transcripts in the brain, neural stem/progenitor cells and P19 cells, suggesting they have an important role in cerebral corticogenesis and neuronal/glial cell differentiation

Erratum to: Methods for evaluating medical tests and biomarkers

[This corrects the article DOI: 10.1186/s41512-016-0001-y.]

Evidence synthesis to inform model-based cost-effectiveness evaluations of diagnostic tests: a methodological systematic review of health technology assessments

Background: Evaluations of diagnostic tests are challenging because of the indirect nature of their impact on patient outcomes. Model-based health economic evaluations of tests allow different types of evidence from various sources to be incorporated and enable cost-effectiveness estimates to be made beyond the duration of available study data. To parameterize a health-economic model fully, all the ways a test impacts on patient health must be quantified, including but not limited to diagnostic test accuracy. Methods: We assessed all UK NIHR HTA reports published May 2009-July 2015. Reports were included if they evaluated a diagnostic test, included a model-based health economic evaluation and included a systematic review and meta-analysis of test accuracy. From each eligible report we extracted information on the following topics: 1) what evidence aside from test accuracy was searched for and synthesised, 2) which methods were used to synthesise test accuracy evidence and how did the results inform the economic model, 3) how/whether threshold effects were explored, 4) how the potential dependency between multiple tests in a pathway was accounted for, and 5) for evaluations of tests targeted at the primary care setting, how evidence from differing healthcare settings was incorporated. Results: The bivariate or HSROC model was implemented in 20/22 reports that met all inclusion criteria. Test accuracy data for health economic modelling was obtained from meta-analyses completely in four reports, partially in fourteen reports and not at all in four reports. Only 2/7 reports that used a quantitative test gave clear threshold recommendations. All 22 reports explored the effect of uncertainty in accuracy parameters but most of those that used multiple tests did not allow for dependence between test results. 7/22 tests were potentially suitable for primary care but the majority found limited evidence on test accuracy in primary care settings. Conclusions: The uptake of appropriate meta-analysis methods for synthesising evidence on diagnostic test accuracy in UK NIHR HTAs has improved in recent years. Future research should focus on other evidence requirements for cost-effectiveness assessment, threshold effects for quantitative tests and the impact of multiple diagnostic tests

Erratum to: Methods for evaluating medical tests and biomarkers

[This corrects the article DOI: 10.1186/s41512-016-0001-y.]