Impact of funding on biomedical research: a retrospective cohort study

BACKGROUND: Public funding is aimed at facilitating the initiation, completion and publication of research study protocols. However, no evaluation is made to investigate the impact of grant success on the conduct of biomedical research. It is therefore of great interest to compare the fate of funded protocols versus not funded: Are they initiated? Are they completed? Did the results confirm the hypothesis? Were they published? The objective was to investigate the fate of protocols submitted for funding, whether they were funded or not. METHODS: Retrospective cohort study of protocols submitted for funding to the Greater Lyon regional scientific committee in 1997. Initial characteristics of protocols (design, study size, investigator status) were abstracted from archives, and follow-up characteristics (initiation, completion and publication) from a mailed questionnaire to the principal investigators. RESULTS: Among the 142 submitted protocols, follow-up information was available for 114 (80%). As a whole, 38% of studies were funded by the Greater Lyon research committee. The rate of initiation varied from 62% for studies with no acknowledged funding to 100% for studies with both committee and other simultaneous funding. When initiated, the rate of completion was 62% for studies with at least one funding and 40% for studies without acknowledged funding. When completed, publication was reached for 77% of studies with either committee or external funding, for 58% of studies without acknowledged funding and for 37% of studies with both committee and external funding. CONCLUSION: Some protocols submitted for funding were initiated and completed without any funding declared. To our understanding this mean that not all protocols submitted really needed funding and also that health care facilities are unaware that they implicitly financially support and pay for biomedical research

What do the JAMA editors say when they discuss manuscripts that they are considering for publication? Developing a schema for classifying the content of editorial discussion

<p>Abstract</p> <p>Background</p> <p>In an effort to identify previously unrecognized aspects of editorial decision-making, we explored the words and phrases that one group of editors used during their meetings.</p> <p>Methods</p> <p>We performed an observational study of discussions at manuscript meetings at <it>JAMA</it>, a major US general medical journal. One of us (KD) attended 12 editorial meetings in 2003 as a visitor and took notes recording phrases from discussion surrounding 102 manuscripts. In addition, editors attending the meetings completed a form for each manuscript considered, listing the reasons they were inclined to proceed to the next step in publication and reasons they were not (DR attended 4/12 meetings). We entered the spoken and written phrases into NVivo 2.0. We then developed a schema for classifying the editors' phrases, using an iterative approach.</p> <p>Results</p> <p>Our classification schema has three main themes: science, journalism, and writing. We considered 2,463 phrases, of which 87 related mainly to the manuscript topic and were not classified (total 2,376 classified). Phrases related to science predominated (1,274 or 54%). The editors, most of whom were physicians, also placed major weight on goals important to JAMA's mission (journalism goals) such as importance to medicine, strategic emphasis for the journal, interest to the readership, and results (729 or 31% of phrases). About 16% (n = 373) of the phrases used related to writing issues, such as clarity and responses to the referees' comments.</p> <p>Conclusion</p> <p>Classification of editorial discourse provides insight into editorial decision making and concepts that need exploration in future studies.</p

Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias

BACKGROUND: The increased use of meta-analysis in systematic reviews of healthcare interventions has highlighted several types of bias that can arise during the completion of a randomised controlled trial. Study publication bias has been recognised as a potential threat to the validity of meta-analysis and can make the readily available evidence unreliable for decision making. Until recently, outcome reporting bias has received less attention. METHODOLOGY/PRINCIPAL FINDINGS: We review and summarise the evidence from a series of cohort studies that have assessed study publication bias and outcome reporting bias in randomised controlled trials. Sixteen studies were eligible of which only two followed the cohort all the way through from protocol approval to information regarding publication of outcomes. Eleven of the studies investigated study publication bias and five investigated outcome reporting bias. Three studies have found that statistically significant outcomes had a higher odds of being fully reported compared to non-significant outcomes (range of odds ratios: 2.2 to 4.7). In comparing trial publications to protocols, we found that 40-62% of studies had at least one primary outcome that was changed, introduced, or omitted. We decided not to undertake meta-analysis due to the differences between studies. CONCLUSIONS: Recent work provides direct empirical evidence for the existence of study publication bias and outcome reporting bias. There is strong evidence of an association between significant results and publication; studies that report positive or significant results are more likely to be published and outcomes that are statistically significant have higher odds of being fully reported. Publications have been found to be inconsistent with their protocols. Researchers need to be aware of the problems of both types of bias and efforts should be concentrated on improving the reporting of trials

Premature Discontinuation of Prospective Clinical Studies Approved by a Research Ethics Committee - A Comparison of Randomised and Non-Randomised Studies.

Premature discontinuation of clinical studies affects about 25% of randomised controlled trials (RCTs) which raises concerns about waste of scarce resources for research. The risk of discontinuation of non-randomised prospective studies (NPSs) is yet unclear. To compare the proportion of discontinued studies between NPSs and RCTs that received ethical approval. We systematically surveyed prospective longitudinal clinical studies that were approved by a single REC in Freiburg, Germany between 2000 and 2002. We collected study characteristics, identified subsequent publications, and surveyed investigators to elucidate whether a study was discontinued and, if so, why. Of 917 approved studies, 547 were prospective longitudinal studies (306 RCTs and 241 NPSs). NPSs were on average smaller than RCTs, more frequently single centre and pilot studies, and less frequently funded by industry. NPSs were less frequently discontinued than RCTs: 32/221 (14%) versus 78/288 (27%, p&lt;0.001, missing data excluded). Poor recruitment was the most frequent reason for discontinuation in both NPSs (36%) and RCTs (37%). Compared to RCTs, NPSs were at lower risk for discontinuation. Measures to reliably predict, sustain, and stimulate recruitment could prevent discontinuation of many RCTs but also of some NPSs

Extent of non-publication in cohorts of studies approved by research ethics committees or included in trial registries

BACKGROUND: The synthesis of published research in systematic reviews is essential when providing evidence to inform clinical and health policy decision-making. However, the validity of systematic reviews is threatened if journal publications represent a biased selection of all studies that have been conducted (dissemination bias). To investigate the extent of dissemination bias we conducted a systematic review that determined the proportion of studies published as peer-reviewed journal articles and investigated factors associated with full publication in cohorts of studies (i) approved by research ethics committees (RECs) or (ii) included in trial registries. METHODS AND FINDINGS: Four bibliographic databases were searched for methodological research projects (MRPs) without limitations for publication year, language or study location. The searches were supplemented by handsearching the references of included MRPs. We estimated the proportion of studies published using prediction intervals (PI) and a random effects meta-analysis. Pooled odds ratios (OR) were used to express associations between study characteristics and journal publication. Seventeen MRPs (23 publications) evaluated cohorts of studies approved by RECs; the proportion of published studies had a PI between 22% and 72% and the weighted pooled proportion when combining estimates would be 46.2% (95% CI 40.2%-52.4%, I2 = 94.4%). Twenty-two MRPs (22 publications) evaluated cohorts of studies included in trial registries; the PI of the proportion published ranged from 13% to 90% and the weighted pooled proportion would be 54.2% (95% CI 42.0%-65.9%, I2 = 98.9%). REC-approved studies with statistically significant results (compared with those without statistically significant results) were more likely to be published (pooled OR 2.8; 95% CI 2.2-3.5). Phase-III trials were also more likely to be published than phase II trials (pooled OR 2.0; 95% CI 1.6-2.5). The probability of publication within two years after study completion ranged from 7% to 30%. CONCLUSIONS: A substantial part of the studies approved by RECs or included in trial registries remains unpublished. Due to the large heterogeneity a prediction of the publication probability for a future study is very uncertain. Non-publication of research is not a random process, e.g., it is associated with the direction of study findings. Our findings suggest that the dissemination of research findings is biased

Reporting of clinical trials: a review of research funders' guidelines

BACKGROUND: Randomised controlled trials (RCTs) represent the gold standard methodological design to evaluate the effectiveness of an intervention in humans but they are subject to bias, including study publication bias and outcome reporting bias. National and international organisations and charities give recommendations for good research practice in relation to RCTs but to date no review of these guidelines has been undertaken with respect to reporting bias. METHODS: National and international organisations and UK based charities listed on the Association for Medical Research Charities website were contacted in 2007; they were considered eligible for this review if they funded RCTs. Guidelines were obtained and assessed in relation to what was written about trial registration, protocol adherence and trial publication. It was also noted whether any monitoring against these guidelines was undertaken. This information was necessary to discover how much guidance researchers are given on the publication of results, in order to prevent study publication bias and outcome reporting bias. RESULTS: Seventeen organisations and 56 charities were eligible of 140 surveyed for this review, although there was no response from 12. Trial registration, protocol adherence, trial publication and monitoring against the guidelines were often explicitly discussed or implicitly referred too. However, only eleven of these organisations or charities mentioned the publication of negative as well as positive outcomes and just three of the organisations specifically stated that the statistical analysis plan should be strictly adhered to and all changes should be reported. CONCLUSION: Our review indicates that there is a need to provide more detailed guidance for those conducting and reporting clinical trials to help prevent the selective reporting of results. Statements found in the guidelines generally refer to publication bias rather than outcome reporting bias. Current guidelines need to be updated and include the statement that all primary and secondary outcomes prespecified in the protocol should be fully reported and should not be selected for inclusion in the final report based on their results

Trial Publication after Registration in ClinicalTrials.Gov: A Cross-Sectional Analysis

Joseph Ross and colleagues examine publication rates of clinical trials and find low rates of publication even following registration in Clinicaltrials.gov

Likely country of origin in publications on randomised controlled trials and controlled clinical trials during the last 60 years

BACKGROUND: The number of publications on clinical trials is unknown as well as the countries publishing most trial reports. To try to examine these questions we performed an ecological study. METHODS: We searched the 454,449 records on publications in The Cochrane Central Register of Controlled Trials (CENTRAL) in The Cochrane Library, Issue 3, 2005 (CD-ROM version) for possible country of origin. We inspected a random sample of 906 records for information on country and type of trial. RESULTS: There was an exponential growth of publications on randomised controlled trials and controlled clinical trials since 1946, but the growth seems to have seized since 2000. We identified the possible country of origin of 210,974 publications (46.4%). The USA is leading with about 46,789 publications followed by UK, Germany, Italy, the Netherlands, Canada, and France. Sweden becomes the leader with 891 publications per million inhabitants during the last 60 years followed by Denmark (n = 864), New Zealand (n = 791), Finland (n = 781), the Netherlands (n = 570), Switzerland (n = 547), and Norway (n = 543). In depth assessment of the random sample backed these findings. CONCLUSION: Many records lacked country of origin, even after the additional scrutiny. The number of publications on clinical trials increased exponentially until the turn of the century. Rather small, democratic, and wealthy countries take the lead when the number of publications on clinical trials is calculated based on million inhabitants. If all countries produced the same number of trials as these countries, this could mean thousands of new effective treatments during the next 60 years

Completeness and Changes in Registered Data and Reporting Bias of Randomized Controlled Trials in ICMJE Journals after Trial Registration Policy

We assessed the adequacy of randomized controlled trial (RCT) registration, changes to registration data and reporting completeness for articles in ICMJE journals during 2.5 years after registration requirement policy.For a set of 149 reports of 152 RCTs with ClinicalTrials.gov registration number, published from September 2005 to April 2008, we evaluated the completeness of 9 items from WHO 20-item Minimum Data Set relevant for assessing trial quality. We also assessed changes to the registration elements at the Archive site of ClinicalTrials.gov and compared published and registry data.RCTs were mostly registered before 13 September 2005 deadline (n = 101, 66.4%); 118 (77.6%) started recruitment before and 31 (20.4%) after registration. At the time of registration, 152 RCTs had a total of 224 missing registry fields, most commonly 'Key secondary outcomes' (44.1% RCTs) and 'Primary outcome' (38.8%). More RCTs with post-registration recruitment had missing Minimum Data Set items than RCTs with pre-registration recruitment: 57/118 (48.3%) vs. 24/31 (77.4%) (χ(2) (1) = 7.255, P = 0.007). Major changes in the data entries were found for 31 (25.2%) RCTs. The number of RCTs with differences between registered and published data ranged from 21 (13.8%) for Study type to 118 (77.6%) for Target sample size.ICMJE journals published RCTs with proper registration but the registration data were often not adequate, underwent substantial changes in the registry over time and differed in registered and published data. Editors need to establish quality control procedures in the journals so that they continue to contribute to the increased transparency of clinical trials

An eight-year follow-up national study of medical school and general hospital ethics committees in Japan

<p>Abstract</p> <p>Background</p> <p>Ethics committees and their system of research protocol peer-review are currently used worldwide. To ensure an international standard for research ethics and safety, however, data is needed on the quality and function of each nation's ethics committees. The purpose of this study was to describe the characteristics and developments of ethics committees established at medical schools and general hospitals in Japan.</p> <p>Methods</p> <p>This study consisted of four national surveys sent twice over a period of eight years to two separate samples. The first target was the ethics committees of all 80 medical schools and the second target was all general hospitals with over 300 beds in Japan (n = 1457 in 1996 and n = 1491 in 2002). Instruments contained four sections: (1) committee structure, (2) frequency of annual meetings, (3) committee function, and (4) existence of a set of guidelines for the refusal of blood transfusion by Jehovah's Witnesses.</p> <p>Results</p> <p>Committee structure was overall interdisciplinary. Frequency of annual meetings increased significantly for both medical school and hospital ethics committees over the eight years. The primary activities for medical school and hospital ethics committees were research protocol reviews and policy making. Results also showed a significant increase in the use of ethical guidelines, particularly those related to the refusal of blood transfusion by Jehovah's Witnesses, among both medical school and hospital ethics committees.</p> <p>Conclusion</p> <p>Overall findings indicated a greater recognized degree of responsibilities and an increase in workload for Japanese ethics committees.</p