Trial Registration for Public Trust: Making the Case for Medical Devices

Recently, several pharmaceutical companies have been shown to have withheld negative clinical trial results from the public. These incidents have resulted in a concerted global effort to register all trials at inception, so that all subsequent results can be tracked regardless of whether they are positive or negative. These trial registration policies have been driven in large part by concern about the pharmaceutical sector. The medical device industry is much smaller, and different from the pharmaceutical industry in some fundamental ways. This paper examines the issues surrounding registration of device trials and argues that these differences with pharmaceutical should not exempt device trials from registration

Bibliometrics of systematic reviews : analysis of citation rates and journal impact factors

Background: Systematic reviews are important for informing clinical practice and health policy. The aim of this study was to examine the bibliometrics of systematic reviews and to determine the amount of variance in citations predicted by the journal impact factor (JIF) alone and combined with several other characteristics. Methods: We conducted a bibliometric analysis of 1,261 systematic reviews published in 2008 and the citations to them in the Scopus database from 2008 to June 2012. Potential predictors of the citation impact of the reviews were examined using descriptive, univariate and multiple regression analysis. Results: The mean number of citations per review over four years was 26.5 (SD +/-29.9) or 6.6 citations per review per year. The mean JIF of the journals in which the reviews were published was 4.3 (SD +/-4.2). We found that 17% of the reviews accounted for 50% of the total citations and 1.6% of the reviews were not cited. The number of authors was correlated with the number of citations (r = 0.215, P =5.16) received citations in the bottom quartile (eight or fewer), whereas 9% of reviews published in the lowest JIF quartile (<=2.06) received citations in the top quartile (34 or more). Six percent of reviews in journals with no JIF were also in the first quartile of citations. Conclusions: The JIF predicted over half of the variation in citations to the systematic reviews. However, the distribution of citations was markedly skewed. Some reviews in journals with low JIFs were well-cited and others in higher JIF journals received relatively few citations; hence the JIF did not accurately represent the number of citations to individual systematic reviews

International ranking systems for universities and institutions: a critical appraisal

<p>Abstract</p> <p>Background</p> <p>Ranking of universities and institutions has attracted wide attention recently. Several systems have been proposed that attempt to rank academic institutions worldwide.</p> <p>Methods</p> <p>We review the two most publicly visible ranking systems, the Shanghai Jiao Tong University 'Academic Ranking of World Universities' and the Times Higher Education Supplement 'World University Rankings' and also briefly review other ranking systems that use different criteria. We assess the construct validity for educational and research excellence and the measurement validity of each of the proposed ranking criteria, and try to identify generic challenges in international ranking of universities and institutions.</p> <p>Results</p> <p>None of the reviewed criteria for international ranking seems to have very good construct validity for both educational and research excellence, and most don't have very good construct validity even for just one of these two aspects of excellence. Measurement error for many items is also considerable or is not possible to determine due to lack of publication of the relevant data and methodology details. The concordance between the 2006 rankings by Shanghai and Times is modest at best, with only 133 universities shared in their top 200 lists. The examination of the existing international ranking systems suggests that generic challenges include adjustment for institutional size, definition of institutions, implications of average measurements of excellence versus measurements of extremes, adjustments for scientific field, time frame of measurement and allocation of credit for excellence.</p> <p>Conclusion</p> <p>Naïve lists of international institutional rankings that do not address these fundamental challenges with transparent methods are misleading and should be abandoned. We make some suggestions on how focused and standardized evaluations of excellence could be improved and placed in proper context.</p

Contribution of Case Reports to Brain Metastases Research: Systematic Review and Analysis of Pattern of Citation

Research activity related to different aspects of prevention, prediction, diagnosis and treatment of brain metastases has increased during recent years. One of the major databases (Scopus) contains 942 scientific articles that were published during the 5-year time period 2006–2010. Of these, 195 (21%) reported on single patient cases and 12 (1%) were reports of 2 cases. Little is known about their influence on advancement of the field or scientific merits. Do brain metastases case reports attract attention and provide stimuli for further research or do they go largely unrecognized? Different measures of impact, visibility and quality of published research are available, each with its own pros and cons. For the present evaluation, article citation rate was chosen. The median number of citations overall and stratified by year of publication was 0, except for the year 2006 when it was 2. As compared to other articles, case reports remained more often without citation (p<0.05 except for 2006 data). All case reports with 10 or more citations (n = 6) reported on newly introduced anticancer drugs, which commonly are prescribed to treat extracranial metastases, and the responses observed in single patients with brain metastases. Average annual numbers of citations were also calculated. The articles with most citations per year were the same six case reports mentioned above (the only ones that obtained more than 2.0 citations per year). Citations appeared to gradually increase during the first two years after publication but remained on a generally low or modest level. It cannot be excluded that case reports without citation provide interesting information to some clinicians or researchers. Apparently, case reports describing unexpected therapeutic success gain more attention, at least in terms of citation, than others

Interpreting systematic reviews: are we ready to make our own conclusions? A cross-sectional study

<p>Abstract</p> <p>Background</p> <p>Independent evaluation of clinical evidence is advocated in evidence-based medicine (EBM). However, authors' conclusions are often appealing for readers who look for quick messages. We assessed how well a group of Malaysian hospital practitioners and medical students derived their own conclusions from systematic reviews (SRs) and to what extent these were influenced by their prior beliefs and the direction of the study results.</p> <p>Methods</p> <p>We conducted two cross-sectional studies: one with hospital practitioners (<it>n </it>= 150) attending an EBM course in June 2008 in a tertiary hospital and one with final-year medical students (<it>n </it>= 35) in November 2008. We showed our participants four Cochrane SR abstracts without the authors' conclusions. For each article, the participants chose a conclusion from among six options comprising different combinations of the direction of effect and the strength of the evidence. We predetermined the single option that best reflected the actual authors' conclusions and labelled this as our best conclusion. We compared the participants' choices with our predetermined best conclusions. Two chosen reviews demonstrated that the intervention was beneficial ("positive"), and two others did not ("negative"). We also asked the participants their prior beliefs about the intervention.</p> <p>Results</p> <p>Overall, 60.3% correctly identified the direction of effect, and 30.1% chose the best conclusions, having identified both the direction of effect and the strength of evidence. More students (48.2%) than practitioners (22.2%) chose the best conclusions (<it>P </it>< 0.001). Fewer than one-half (47%) correctly identified the direction of effect against their prior beliefs. "Positive" SRs were more likely than "negative" SRs to change the participants' beliefs about the effect of the intervention (relative risk (RR) 1.8, 95% confidence interval 1.3 to 2.6) and "convert" those who were previously unsure by making them choose the appropriate direction of effect (RR 1.9, 95% confidence interval 1.3 to 2.8).</p> <p>Conclusions</p> <p>The majority of our participants could not generate appropriate conclusions from SRs independently. Judicious direction from the authors' conclusions still appears crucial to guiding our health care practitioners in identifying appropriate messages from research. Authors, editors and reviewers should ensure that the conclusions of a paper accurately reflect the results. Similar studies should be conducted in other settings where awareness and application of EBM are different.</p> <p>Please see Commentary: <url>http://www.biomedcentral.com/1741-7015/9/31/</url>.</p

Characteristics Associated with Citation Rate of the Medical Literature

BACKGROUND: The citation rate for articles is viewed as a measure of their importance and impact; however, little is known about what features of articles are associated with higher citation rate. METHODOLOGY/PRINCIPAL FINDINGS: We conducted a cohort study of all original articles, regardless of study methodology, published in the Lancet, JAMA, and New England Journal of Medicine, from October 1, 1999 to March 31, 2000. We identified 328 articles. Two blinded, independent reviewers extracted, in duplicate, nine variables from each article, which were analyzed in both univariable and multivariable linear least-squares regression models for their association with the annual rate of citations received by the article since publication. A two-way interaction between industry funding and an industry-favoring result was tested and found to be significant (p = 0.02). In our adjusted analysis, the presence of industry funding and an industry-favoring result was associated with an increase in annual citation rate of 25.7 (95% confidence interval, 8.5 to 42.8) compared to the absence of both industry funding and industry-favoring results. Higher annual rates of citation were also associated with articles dealing with cardiovascular medicine (13.3 more; 95% confidence interval, 3.9 to 22.3) and oncology (12.6 more; 95% confidence interval, 1.2 to 24.0), articles with group authorship (11.1 more; 95% confidence interval, 2.7 to 19.5), larger sample size and journal of publication. CONCLUSIONS/SIGNIFICANCE: Large trials, with group authorship, industry-funded, with industry-favoring results, in oncology or cardiology were associated with greater subsequent citations

Financial considerations in the conduct of multi-centre randomised controlled trials: evidence from a qualitative study.

National Coordinating Centre for Research Methodology; Medical Research Council, UK Department of Health; Chief Scientist OfficeNot peer reviewedPublisher PD