Reducing the Probability of False Positive Research Findings by Pre-Publication Validation - Experience with a Large Multiple Sclerosis Database

*Objective*
We have assessed the utility of a pre-publication validation policy in reducing the probability of publishing false positive research findings. 
*Study design and setting*
The large database of the Sylvia Lawry Centre for Multiple Sclerosis Research was split in two parts: one for hypothesis generation and a validation part for confirmation of selected results. We present case studies from 5 finalized projects that have used the validation policy and results from a simulation study.
*Results*
In one project, the "relapse and disability" project as described in section II (example 3), findings could not be confirmed in the validation part of the database. The simulation study showed that the percentage of false positive findings can exceed 20% depending on variable selection. 
*Conclusion*
We conclude that the validation policy has prevented the publication of at least one research finding that could not be validated in an independent data set (and probably would have been a "true" false-positive finding) over the past three years, and has led to improved data analysis, statistical programming, and selection of hypotheses. The advantages outweigh the lost statistical power inherent in the process

Tract-based spatial statistics to assess the neuroprotective effect of early erythropoietin on white matter development in preterm infants

In a randomized double-blind placebo-controlled trial, O'Gorman et al. show that administration of erythropoietin within the first 42 hours after preterm birth improves white matter development in preterm infants. Improvements are seen in the corpus callosum, the anterior and posterior limbs of the internal capsule, and the corticospinal tract bilaterall

Reducing the probability of false positive research findings by pre-publication validation – Experience with a large multiple sclerosis database

<p>Abstract</p> <p>Background</p> <p>Published false positive research findings are a major problem in the process of scientific discovery. There is a high rate of lack of replication of results in clinical research in general, multiple sclerosis research being no exception. Our aim was to develop and implement a policy that reduces the probability of publishing false positive research findings.</p> <p>We have assessed the utility to work with a pre-publication validation policy after several years of research in the context of a large multiple sclerosis database.</p> <p>Methods</p> <p>The large database of the Sylvia Lawry Centre for Multiple Sclerosis Research was split in two parts: one for hypothesis generation and a validation part for confirmation of selected results. We present case studies from 5 finalized projects that have used the validation policy and results from a simulation study.</p> <p>Results</p> <p>In one project, the "relapse and disability" project as described in section II (example 3), findings could not be confirmed in the validation part of the database. The simulation study showed that the percentage of false positive findings can exceed 20% depending on variable selection.</p> <p>Conclusion</p> <p>We conclude that the validation policy has prevented the publication of at least one research finding that could not be validated in an independent data set (and probably would have been a "true" false-positive finding) over the past three years, and has led to improved data analysis, statistical programming, and selection of hypotheses. The advantages outweigh the lost statistical power inherent in the process.</p

Hybridization thermodynamics of NimbleGen Microarrays

Background While microarrays are the predominant method for gene expression profiling, probe signal variation is still an area of active research. Probe signal is sequence dependent and affected by probe-target binding strength and the competing formation of probe-probe dimers and secondary structures in probes and targets. Results We demonstrate the benefits of an improved model for microarray hybridization and assess the relative contributions of the probe-target binding strength and the different competing structures. Remarkably, specific and unspecific hybridization were apparently driven by different energetic contributions: For unspecific hybridization, the melting temperature Tm was the best predictor of signal variation. For specific hybridization, however, the effective interaction energy that fully considered competing structures was twice as powerful a predictor of probe signal variation. We show that this was largely due to the effects of secondary structures in the probe and target molecules. The predictive power of the strength of these intramolecular structures was already comparable to that of the melting temperature or the free energy of the probe-target duplex. Conclusions This analysis illustrates the importance of considering both the effects of probe-target binding strength and the different competing structures. For specific hybridization, the secondary structures of probe and target molecules turn out to be at least as important as the probe-target binding strength for an understanding of the observed microarray signal intensities. Besides their relevance for the design of new arrays, our results demonstrate the value of improving thermodynamic models for the read-out and interpretation of microarray signals

Vignette studies of medical choice and judgement to study caregivers' medical decision behaviour: systematic review

BACKGROUND: Vignette studies of medical choice and judgement have gained popularity in the medical literature. Originally developed in mathematical psychology they can be used to evaluate physicians' behaviour in the setting of diagnostic testing or treatment decisions. We provide an overview of the use, objectives and methodology of these studies in the medical field. METHODS: Systematic review. We searched in electronic databases; reference lists of included studies. We included studies that examined medical decisions of physicians, nurses or medical students using cue weightings from answers to structured vignettes. Two reviewers scrutinized abstracts and examined full text copies of potentially eligible studies. The aim of the included studies, the type of clinical decision, the number of participants, some technical aspects, and the type of statistical analysis were extracted in duplicate and discrepancies were resolved by consensus. RESULTS: 30 reports published between 1983 and 2005 fulfilled the inclusion criteria. 22 studies (73%) reported on treatment decisions and 27 (90%) explored the variation of decisions among experts. Nine studies (30%) described differences in decisions between groups of caregivers and ten studies (33%) described the decision behaviour of only one group. Only six studies (20%) compared decision behaviour against an empirical reference of a correct decision. The median number of considered attributes was 6.5 (IQR 4-9), the median number of vignettes was 27 (IQR 16-40). In 17 studies, decision makers had to rate the relative importance of a given vignette; in six studies they had to assign a probability to each vignette. Only ten studies (33%) applied a statistical procedure to account for correlated data. CONCLUSION: Various studies of medical choice and judgement have been performed to depict weightings of the value of clinical information from answers to structured vignettes of care givers. We found that the design and analysis methods used in current applications vary considerably and could be improved in a large number of cases

Tract-based spatial statistics to assess the neuroprotective effect of early erythropoietin on white matter development in preterm infants

Despite improved survival, many preterm infants undergo subsequent neurodevelopmental impairment. To date, no neuroprotective therapies have been implemented into clinical practice. Erythropoietin, a haematopoietic cytokine used for treatment of anaemia of prematurity, has been shown to have neuroprotective and neuroregenerative effects on the brain in many experimental studies. The aim of the study was to assess the effect of recombinant human erythropoietin on the microstructural development of the cerebral white matter using tract-based spatial statistics performed at term equivalent age. A randomized, double-blind placebo-controlled, prospective multicentre study applying recombinant human erythropoietin in the first 42 h after preterm birth entitled 'Does erythropoietin improve outcome in preterm infant' was conducted in Switzerland (NCT00413946). Preterm infants were given recombinant human erythropoietin (3000 IU) or an equivalent volume of placebo (NaCl 0.9%) intravenously before 3 h of age after birth, at 12-18 h and at 36-42 h after birth. High resolution diffusion tensor imaging was obtained at 3 T in 58 preterm infants with mean (standard deviation) gestational age at birth 29.75 (1.44) weeks, and at scanning at 41.1 (2.09) weeks. Imaging was performed at a single centre. Voxel-wise statistical analysis of the fractional anisotropy data was carried out using tract-based spatial statistics to test for differences in fractional anisotropy between infants treated with recombinant human erythropoietin and placebo using a general linear model, covarying for the gestational age at birth and the corrected gestational age at the time of the scan. Preterm infants treated with recombinant human erythropoietin demonstrated increased fractional anisotropy in the genu and splenium of the corpus callosum, the anterior and posterior limbs of the internal capsule, and the corticospinal tract bilaterally. Mean fractional anisotropy was significantly higher in preterm infants treated with recombinant human erythropoietin than in those treated with placebo (P < 0.001). We conclude that early recombinant human erythropoietin administration improves white matter development in preterm infants assessed by diffusion tensor imaging and tract-based spatial statistics