How to combine rules and commitment in fostering research integrity?

Research integrity (RI) is crucial for producing research that is trustworthy and of high quality. Rules are important in setting RI standards, improving research practice and fostering responsible research practices. At the same time, rules can lead to increased bureaucracy, which without commensurate increased commitment amongst researchers towards RI is unlikely to lead to more responsible research behavior. In this paper, we explore the question: How can rules and commitment be combined to foster RI? There are three ways that research institutions can govern RI: markets (governing through incentives), hierarchies or bureaucracies (governing through rules), and network processes (governing through commitment and agreement at group level). Based on Habermas’ Theory of Communicative Action, we argue that network processes focusing on consensus, as part of the lifeworld, are necessary to legitimize and support systems, i.e. market and bureaucratic modes of governance. We analyze the institutional response to a serious RI case to illustrate how network processes can create a context in which rules can foster RI. Specifically, we analyze how the Science Committee established at Tilburg University in 2012 has navigated and combined different modes of governance to foster RI. Based on this case analysis, we formulate recommendations to research institutions on how to combine rules and commitment

Meta-analysis: Acupuncture for osteoarthritis of the knee

Background: Knee osteoarthritis is a major cause of pain and functional limitation. Purpose: To evaluate the effects of acupuncture for treating knee osteoarthritis. Data Sources: Cochrane Central Register of Controlled Trials, MEDLINE, and EMBASE databases to January 2007. No language restrictions were applied. Study Selection: Randomized trials longer than 6 weeks in duration that compared needle acupuncture with a sham, usual care, or waiting list control group for patients with knee osteoarthritis. Data Extraction: Two authors independently agreed on eligibility, assessed methodological quality and acupuncture adequacy, and extracted outcome data on pain and function measures. Data Synthesis: Eleven trials met the selection criteria, and 9 reported sufficient data for pooling. Standardized mean differences were calculated by using differences in improvements from baseline between patients assigned to acupuncture and those assigned to control groups. Compared with patients in waiting list control groups, patients who received acupuncture reported clinically relevant short-term improvements in pain (standardized mean difference, -0.96 [95% CI, -1.21 to -0.70]) and function (standardized mean difference, -0.93 [CI, -1.16 to -0.69]). Patients who received acupuncture also reported clinically relevant short- and long-term improvements in pain and function compared with patients in usual care control groups. Compared with a sham control, acupuncture provided clinically irrelevant short-term improvements in pain (standardized mean difference, -0.35 [CI, -0.55 to -0.15]) and function (standardized mean difference, -0.35 [CI, -0.56 to -0.14]) and clinically irrelevant long-term improvements in pain (standardized mean difference, -0.13 [CI, -0.24 to -0.01]) and function (standardized mean difference, -0.14 [CI, -0.26 to -0.03]). Limitation: Sham-controlled trials had heterogeneous results that were probably due to the variability of acupuncture and sham protocols, patient samples, and settings. Conclusions: Sham-controlled trials show clinically irrelevant short-term benefits of acupuncture for treating knee osteoarthritis. Waiting list-controlled trials suggest clinically relevant benefits, some of which may be due to placebo or expectation effects. © 2007 American College of Physicians.link_to_subscribed_fulltex

Spatial redistribution of irregularly-spaced Pareto fronts for more intuitive navigation and solution selection

A multi-objective optimization approach is o.en followed by an a posteriori decision-making process, during which the most appropriate solution of the Pareto set is selected by a professional in the .eld. Conventional visualization methods do not correct for Pareto fronts with irregularly-spaced solutions. However, achieving a uniform spread of solutions can make the decision-making process more intuitive when decision tools such as sliders, which represent the preference for each objective, are used. We propose a method that maps anm-dimensional Pareto front to an (m-1)-simplex and spreads out points to achieve a more uniform distribution of these points in the simplex while maintaining the local neighborhood structure of the solutions as much as possible. .is set of points can then more intuitively be navigated due to the more uniform distribution. We test our approach on a set of non-uniformly spaced 3D Pareto fronts of a real-world problem: deformable image registration of medical images. The results of these experiments are visualized as points in a triangle, showing that we indeed achieve a representation of the Pareto front with a near-uniform distribution of points where these are still positioned as expected, i.e., according to their quality in each of the objectives of interest

How do scientists perceive the current publication culture? A qualitative focus group interview study among Dutch biomedical researchers.

OBJECTIVE: To investigate the biomedical scientist's perception of the prevailing publication culture. DESIGN: Qualitative focus group interview study. SETTING: Four university medical centres in the Netherlands. PARTICIPANTS: Three randomly selected groups of biomedical scientists (PhD, postdoctoral staff members and full professors). MAIN OUTCOME MEASURES: Main themes for discussion were selected by participants. RESULTS: Frequently perceived detrimental effects of contemporary publication culture were the strong focus on citation measures (like the Journal Impact Factor and the H-index), gift and ghost authorships and the order of authors, the peer review process, competition, the funding system and publication bias. These themes were generally associated with detrimental and undesirable effects on publication practices and on the validity of reported results. Furthermore, senior scientists tended to display a more cynical perception of the publication culture than their junior colleagues. However, even among the PhD students and the postdoctoral fellows, the sentiment was quite negative. Positive perceptions of specific features of contemporary scientific and publication culture were rare. CONCLUSIONS: Our findings suggest that the current publication culture leads to negative sentiments, counterproductive stress levels and, most importantly, to questionable research practices among junior and senior biomedical scientists

External Validation of a Measurement Tool to Assess Systematic Reviews (AMSTAR)

BACKGROUND: Thousands of systematic reviews have been conducted in all areas of health care. However, the methodological quality of these reviews is variable and should routinely be appraised. AMSTAR is a measurement tool to assess systematic reviews. METHODOLOGY: AMSTAR was used to appraise 42 reviews focusing on therapies to treat gastro-esophageal reflux disease, peptic ulcer disease, and other acid-related diseases. Two assessors applied the AMSTAR to each review. Two other assessors, plus a clinician and/or methodologist applied a global assessment to each review independently. CONCLUSIONS: The sample of 42 reviews covered a wide range of methodological quality. The overall scores on AMSTAR ranged from 0 to 10 (out of a maximum of 11) with a mean of 4.6 (95% CI: 3.7 to 5.6) and median 4.0 (range 2.0 to 6.0). The inter-observer agreement of the individual items ranged from moderate to almost perfect agreement. Nine items scored a kappa of >0.75 (95% CI: 0.55 to 0.96). The reliability of the total AMSTAR score was excellent: kappa 0.84 (95% CI: 0.67 to 1.00) and Pearson's R 0.96 (95% CI: 0.92 to 0.98). The overall scores for the global assessment ranged from 2 to 7 (out of a maximum score of 7) with a mean of 4.43 (95% CI: 3.6 to 5.3) and median 4.0 (range 2.25 to 5.75). The agreement was lower with a kappa of 0.63 (95% CI: 0.40 to 0.88). Construct validity was shown by AMSTAR convergence with the results of the global assessment: Pearson's R 0.72 (95% CI: 0.53 to 0.84). For the AMSTAR total score, the limits of agreement were -0.19+/-1.38. This translates to a minimum detectable difference between reviews of 0.64 'AMSTAR points'. Further validation of AMSTAR is needed to assess its validity, reliability and perceived utility by appraisers and end users of reviews across a broader range of systematic reviews

The methodological quality of 176,620 randomized controlled trials published between 1966 and 2018 reveals a positive trend but also an urgent need for improvement

Many randomized controlled trials (RCTs) are biased and difficult to reproduce due to methodological flaws and poor reporting. There is increasing attention for responsible research practices and implementation of reporting guidelines, but whether these efforts have improved the methodological quality of RCTs (e.g., lower risk of bias) is unknown. We, therefore, mapped risk-of-bias trends over time in RCT publications in relation to journal and author characteristics. Meta-information of 176,620 RCTs published between 1966 and 2018 was extracted. The risk-of-bias probability (random sequence generation, allocation concealment, blinding of patients/personnel, and blinding of outcome assessment) was assessed using a risk-of-bias machine learning tool. This tool was simultaneously validated using 63,327 human risk-of-bias assessments obtained from 17,394 RCTs evaluated in the Cochrane Database of Systematic Reviews (CDSR). Moreover, RCT registration and CONSORT Statement reporting were assessed using automated searches. Publication characteristics included the number of authors, journal impact factor (JIF), and medical discipline. The annual number of published RCTs substantially increased over 4 decades, accompanied by increases in authors (5.2 to 7.8) and institutions (2.9 to 4.8). The risk of bias remained present in most RCTs but decreased over time for allocation concealment (63% to 51%), random sequence generation (57% to 36%), and blinding of outcome assessment (58% to 52%). Trial registration (37% to 47%) and the use of the CONSORT Statement (1% to 20%) also rapidly increased. In journals with a higher impact factor (>10), the risk of bias was consistently lower with higher levels of RCT registration and the use of the CONSORT Statement. Automated risk-of-bias predictions had accuracies above 70% for allocation concealment (70.7%), random sequence generation (72.1%), and blinding of patients/personnel (79.8%), but not for blinding of outcome assessment (62.7%). In conclusion, the likelihood of bias in RCTs has generally decreased over the last decades. This optimistic trend may be driven by increased knowledge augmented by mandatory trial registration and more stringent reporting guidelines and journal requirements. Nevertheless, relatively high probabilities of bias remain, particularly in journals with lower impact factors. This emphasizes that further improvement of RCT registration, conduct, and reporting is still urgently needed