Directing scientists away from potentially biased publications:the role of systematic reviews in health care

Despite increasing concerns about the validity of published research, the issue of how the scientific community can maintain a high-quality body of research is not well understood. We consider the case of systematic reviews in health care, and explore whether risk of bias ratings communicated within these reviews may help shift scientists’ attention towards published research that is at a low risk of bias. We focus on publications deemed at risk of bias due to selective reporting; that is, scientific articles with high chances of systematic errors in the published research findings due to flaws in the reporting. Using a matched-sample control group we find that, after potential bias is signalled in systematic reviews, publications at high risk of bias attract less attention – as indicated by fewer follow-on citations – when compared to a control group of low risk of bias publications. We extend our analysis by considering those cases where risk of bias is unclear, and by examining how different features of the rating system may affect the magnitude of the main effect. The findings provide evidence about whether systematic reviews can play a role in signalling biases in the scientific literature, over and above their established role of synthesising prior research

At what point do academics forego citations for journal status?

The limitations of journal based citation metrics for assessing individual researchers are well known. However, the way in which these assessment systems differentially shape research practices within disciplines is less well understood. Presenting evidence from a new analysis of business and management academics, Rossella Salandra and Ammon Salter and James Walker¸ explore how journal status is valued by these academics and the point at which journal status becomes more prized than academic influence

The impact of journal re-grading on perception of ranking systems: exploring the case of the academic journal guide and business and management scholars in the UK

The marked increase in the use of metrics, such as journal lists, to assess research has had a profound effect on academics’ working lives. While some view the diffusion of rankings as beneficial, others consider their diffusion as a malicious development, which further acerbates a tendency towards managerialism in academia, and undermine the integrity and diversity of academic research. Using data from a large-scale survey and a re-grading of journals in a ranking used by Business and Management UK scholars - the Academic Journal Guide (AJG) - as a pseudo-experiment, we examine what determines negative and positive perceptions of rankings. We find that the individuals who published in outlets that were upgraded were less hostile to the ranking than those who did not benefit from these changes, and that individuals were also less hostile to the ranking if outlets in their field had benefited from re-grading in the new list. We also find that the individuals who published in outlets that were upgraded were more positive to the ranking than those who did not benefit from these changes, and that individuals were also more positive to the ranking if outlets in their field had benefited from re-grading in the new list

Disclosing research findings: determinants and impact of incomplete and partial scientific knowledge in the context of evidence-based medicine

This dissertation examines the disclosure of research findings in the context of evidence- based medicine. The determinants and impact of incomplete and partial scientific knowledge are explored in three interrelated studies. Study I (Chapter 4) explores the contextual factors associated with the selective reporting of research findings in clinical trials, defined as the publication of only part of the findings originally recorded during a research study, based on the results. I find that the chances of selective reporting are higher for industry-funded studies than for publicly-funded studies; yet, this effect is restricted to studies where at least one author is industry-affiliated. I also find that selective reporting is more likely in projects exploring radical innovation, compared to those investigating incremental innovation. Study II (Chapter 5) investigates the poor reporting of products’ adverse effects in scientific publications, specifically focussing on the misreporting of drugs’ adverse events in clinical trials. I find support to the hypothesis that negative relative performance at the level of the project increases the likelihood of misrepresentation of adverse effects in the focal project’s final publication. The results also partially support my hypothesis on the influence of competition: in head-to-head competitions (i.e., trials where two drugs are confronting each other), the effect of negative performance on the misreporting of adverse effects is weaker for leaders. Study III (Chapter 6) aims at identifying the causal impact of bias in publication introduced by selective reporting. Using a matched control sample and diff-in-diff methodology, the study suggests that bias (detection) causes a significant decline in future citations. These findings offer support to the view that evidence appraisal systems (e.g., systematic reviews) are effective in signalling incomplete knowledge to the scientific community, in turn contributing to reducing avoidable research waste. The common thread connecting my studies is the investigation of the issues relating to the accuracy and truth of the research record in medical publication; the ‘extreme case’ of medical research speaks to broader challenges currently faced by scientists while they attempt to build a trustworthy knowledge-base, in publications and elsewhere.Open Acces

Exploring preferences for impact versus publications among UK business and management academics

Academics are under increasing pressure to demonstrate the impact of their research with external actors. Some national research assessment systems have mandated academics to document their impact on non-academic actors, and linked research funding to assessments of these impacts. Although there has been considerable debate around the design of these systems, little is known about how academics perceive the value of impact against more conventional academic outputs, such as publications. Using multisource data, including a large-scale survey of UK business and management academics, this paper explores the individual and institutional factors that explain an individual’s preference for impact versus publication. The results show that academics display a preference for impact over publications, even when that impact is not associated with requirements of the assessment system in terms of rigour of the underpinning research. The preference for impact over publications is heightened by organization tenure, non-academic work experience, intrinsic career motivations and research-intensive contexts, while it is weakened by academic influence, extrinsic career motives and academic rank. We explore the implications of these findings for the design of research assessment systems and academics’ reactions to them.<br/

Research project on the impact of Cochrane translation programme

Cochrane is committed to reducing the linguistic barrier to global evidence-informed health decisions. To this end, in 2014, Cochrane adopted a Translation Strategy to support translation activities and make healthcare evidence accessible in different languages. We propose to work with the Cochrane translation programme team to develop a research project to explore the programme's impact