Response to: Pre-referral rectal artesunate in severe malaria: a flawed trial

A response to and comment on Pre-referral rectal artesunate in severe malaria: a flawed trial, by Karim F Hirji and Zulfiqarali G Premji

Readers as research detectives

Flaws in research papers are common but it may require arduous detective work to unravel them. Checklists are helpful, but many inconsistencies will only be revealed through repeated cross-checks of every little detail, just like in a crime case. As a major deterrent for dishonesty, raw data from all trials should be posted on a public website. This would also make it much easier to detect errors and flaws in publications, and it would allow many research projects to be performed without collecting new data. The prevailing culture of secrecy and ownership to data is not in the best interests of patients

Calculating unreported confidence intervals for paired data

<p>Abstract</p> <p>Background</p> <p>Confidence intervals (or associated standard errors) facilitate assessment of the practical importance of the findings of a health study, and their incorporation into a meta-analysis. For paired design studies, these items are often not reported. Since the descriptive statistics for such studies are usually presented in the same way as for unpaired designs, direct computation of the standard error is not possible without additional information.</p> <p>Methods</p> <p>Elementary, well-known relationships between standard errors and <it>p</it>-values were used to develop computation schemes for paired mean difference, risk difference, risk ratio and odds ratio.</p> <p>Results</p> <p>Unreported confidence intervals for large sample paired binary and numeric data can be computed fairly accurately using simple methods provided the <it>p</it>-value is given. In the case of paired binary data, the design based 2 × 2 table can be reconstructed as well.</p> <p>Conclusions</p> <p>Our results will facilitate appropriate interpretation of paired design studies, and their incorporation into meta-analyses.</p

Pre-referral rectal artesunate in severe malaria: flawed trial

<p>Abstract</p> <p>Background</p> <p>Immediate injectable treatment is essential for severe malaria. Otherwise, the afflicted risk lifelong impairment or death. In rural areas of Africa and Asia, appropriate care is often miles away. In 2009, Melba Gomes and her colleagues published the findings of a randomized, placebo-controlled trial of rectal artesunate for suspected severe malaria in such remote areas. Enrolling nearly 18,000 cases, the aim was to evaluate whether, as patients were in transit to a health facility, a pre-referral artesunate suppository blocked disease progression sufficiently to reduce these risks. The affirmative findings of this, the only trial on the issue thus far, have led the WHO to endorse rectal artesunate as a pre-referral treatment for severe malaria. In the light of its public health importance and because its scientific quality has not been assessed for a systematic review, our paper provides a detailed evaluation of the design, conduct, analysis, reporting, and practical features of this trial.</p> <p>Results</p> <p>We performed a checklist-based and an in-depth evaluation of the trial. The evaluation criteria were based on the CONSORT statement for reporting clinical trials, the clinical trial methodology literature, and practice in malaria research. Our main findings are: The inclusion and exclusion criteria and the sample size justification are not stated. Many clearly ineligible subjects were enrolled. The training of the recruiters does not appear to have been satisfactory. There was excessive between center heterogeneity in design and conduct. Outcome evaluation schedule was not defined, and in practice, became too wide. Large gaps in the collection of key data were evident. Primary endpoints were inconsistently utilized and reported; an overall analysis of the outcomes was not done; analyses of time to event data had major flaws; the stated intent-to-treat analysis excluded a third of the randomized subjects; the design-indicated stratified or multi-variate analysis was not done; many improper subgroups were analyzed in a post-hoc fashion; the analysis and reporting metric was deficient. There are concerns relating to patient welfare at some centers. Exclusion of many cases from data analysis compromised external validity. A bias-controlled reanalysis of available data does not lend support to the conclusions drawn by the authors.</p> <p>Conclusions</p> <p>This trial has numerous serious deficiencies in design, implementation, and methods of data analysis. Interpretation and manner of reporting are wanting, and the applicability of the findings is unclear. The trial conduct could have been improved to better protect patient welfare. The totality of these problems make it a flawed study whose conclusions remain subject to appreciable doubt.</p

Outcome based subgroup analysis: a neglected concern

A subgroup of clinical trial subjects identified by baseline characteristics is a proper subgroup while a subgroup determined by post randomization events or measures is an improper subgroup. Both types of subgroups are often analyzed in clinical trial papers. Yet, the extensive scrutiny of subgroup analyses has almost exclusively attended to the former. The analysis of improper subgroups thereby not only flourishes in numerous disguised ways but also does so without a corresponding awareness of its pitfalls. Comparisons of the grade of angina in a heart disease trial, for example, usually include only the survivors. This paper highlights some of the distinct ways in which outcome based subgroup analysis occurs, describes the hazards associated with it, and proposes a simple alternative approach to counter its analytic bias. Data from six published trials show that outcome based subgroup analysis, like proper subgroup analysis, may be performed in a post-hoc fashion, overdone, selectively reported, and over interpreted. Six hypothetical trial scenarios illustrate the forms of hidden bias related to it. That bias can, however, be addressed by assigning clinically appropriate scores to the usually excluded subjects and performing an analysis that includes all the randomized subjects. A greater level of awareness about the practice and pitfalls of outcome based subgroup analysis is needed. When required, such an analysis should maintain the integrity of randomization. This issue needs greater practical and methodologic attention than has been accorded to it thus far

Reporting on covariate adjustment in randomised controlled trials before and after revision of the 2001 CONSORT statement: a literature review

<p>Abstract</p> <p>Objectives</p> <p>To evaluate the use and reporting of adjusted analysis in randomised controlled trials (RCTs) and compare the quality of reporting before and after the revision of the CONSORT Statement in 2001.</p> <p>Design</p> <p>Comparison of two cross sectional samples of published articles.</p> <p>Data Sources</p> <p>Journal articles indexed on PubMed in December 2000 and December 2006.</p> <p>Study Selection</p> <p>Parallel group RCTs with a full publication carried out in humans and published in English</p> <p>Main outcome measures</p> <p>Proportion of articles reported adjusted analysis; use of adjusted analysis; the reason for adjustment; the method of adjustment and the reporting of adjusted analysis results in the main text and abstract.</p> <p>Results</p> <p>In both cohorts, 25% of studies reported adjusted analysis (84/355 in 2000 vs 113/422 in 2006). Compared with articles reporting only unadjusted analyses, articles that reported adjusted analyses were more likely to specify primary outcomes, involve multiple centers, perform stratified randomization, be published in general medical journals, and recruit larger sample sizes. In both years a minority of articles explained why and how covariates were selected for adjustment (20% to 30%). Almost all articles specified the statistical methods used for adjustment (99% in 2000 vs 100% in 2006) but only 5% and 10%, respectively, reported both adjusted and unadjusted results as recommended in the CONSORT guidelines.</p> <p>Conclusion</p> <p>There was no evidence of change in the reporting of adjusted analysis results five years after the revision of the CONSORT Statement and only a few articles adhered fully to the CONSORT recommendations.</p

No short-cut in assessing trial quality: a case study

Assessing the quality of included trials is a central part of a systematic review. Many check-list type of instruments for doing this exist. Using a trial of antibiotic treatment for acute otitis media, Burke et al., BMJ, 1991, as the case study, this paper illustrates some limitations of the check-list approach to trial quality assessment. The general verdict from the check list type evaluations in nine relevant systematic reviews was that Burke et al. (1991) is a good quality trial. All relevant meta-analyses extensively used its data to formulate therapeutic evidence. My comprehensive evaluation, on the other hand, brought to the surface a series of serious problems in the design, conduct, analysis and report of this trial that were missed by the earlier evaluations. A check-list or instrument based approach, if used as a short-cut, may at times rate deeply flawed trials as good quality trials. Check lists are crucial but they need to be augmented with an in-depth review, and where possible, a scrutiny of the protocol, trial records, and original data. The extent and severity of the problems I uncovered for this particular trial warrant an independent audit before it is included in a systematic review

Sequence variations of ABCB1, SLC6A2, SLC6A3, SLC6A4, CREB1, CRHR1 and NTRK2: association with major depression and antidepressant response in Mexican-Americans

We studied seven genes that reflect events relevant to antidepressant action at four sequential levels: (1) entry into the brain, (2) binding to monoaminergic transporters, and (3) distal effects at the transcription level, resulting in (4) changes in neurotrophin and neuropeptide receptors. Those genes are ATP-binding cassette subfamily B member 1 (ABCB1), the noradrenaline, dopamine, and serotonin transporters (SLC6A2, SLC6A3 and SLC6A4), cyclic AMP-responsive element binding protein 1 (CREB1), corticotropin-releasing hormone receptor 1 (CRHR1) and neurotrophic tyrosine kinase type 2 receptor (NTRK2). Sequence variability for those genes was obtained in exonic and flanking regions. A total of 56 280 000 bp across were sequenced in 536 unrelated Mexican Americans from Los Angeles (264 controls and 272 major depressive disorder (MDD)). We detected in those individuals 419 single nucleotide polymorphisms (SNPs); the nucleotide diversity was 0.00054±0.0001. Of those, a total of 204 novel SNPs were identified, corresponding to 49% of all previously reported SNPs in those genes: 72 were in untranslated regions, 19 were in coding sequences of which 7 were non-synonymous, 86 were intronic and 27 were in upstream/downstream regions. Several SNPs or haplotypes in ABCB1, SLC6A2, SLC6A3, SLC6A4, CREB1 and NTRK2 were associated with MDD, and in ABCB1, SLC6A2 and NTRK2 with antidepressant response. After controlling for age, gender and baseline 21-item Hamilton Depression Rating Scale (HAM-D21) score, as well as correcting for multiple testing, the relative reduction of HAM-D21 score remained significantly associated with two NTRK2-coding SNPs (rs2289657 and rs56142442) and the haplotype CAG at rs2289658 (splice site), rs2289657 and rs2289656. Further studies in larger independent samples will be needed to confirm these associations. Our data indicate that extensive assessment of sequence variability may contribute to increase understanding of disease susceptibility and drug response. Moreover, these results highlight the importance of direct re-sequencing of key candidate genes in ethnic minority groups in order to discover novel genetic variants that cannot be simply inferred from existing databases