The evaluation scale:exploring decisions about societal impact in peer review panels

Realising the societal gains from publicly funded health and medical research requires a model for a reflexive evaluation precedent for the societal impact of research. This research explores UK Research Excellence Framework evaluators’ values and opinions and assessing societal impact, prior to the assessment taking place. Specifically, we discuss the characteristics of two different impact assessment extremes – the “quality-focused” evaluation and “societal impact-focused” evaluation. We show the wide range of evaluator views about impact, and that these views could be conceptually reflected in a range of different positions along a conceptual evaluation scale. We describe the characteristics of these extremes in detail, and discuss the different beliefs evaluators had which could influence where they positioned themselves along the scale. These decisions, we argue, when considered together, form a dominant definition of societal impact that influences the direction of its evaluation by the panel

The Association between Four Citation Metrics and Peer Rankings of Research Influence of Australian Researchers in Six Fields of Public Health

Doubt about the relevance, appropriateness and transparency of peer review has promoted the use of citation metrics as a viable adjunct or alternative in the assessment of research impact. It is also commonly acknowledged that research metrics will not replace peer review unless they are shown to correspond with the assessment of peers. This paper evaluates the relationship between researchers' influence as evaluated by their peers and various citation metrics representing different aspects of research output in 6 fields of public health in Australia. For four fields, the results showed a modest positive correlation between different research metrics and peer assessments of research influence. However, for two fields, tobacco and injury, negative or no correlations were found. This suggests a peer understanding of research influence within these fields differed from visibility in the mainstream, peer-reviewed scientific literature. This research therefore recommends the use of both peer review and metrics in a combined approach in assessing research influence. Future research evaluation frameworks intent on incorporating metrics should first analyse each field closely to determine what measures of research influence are valued highly by members of that research community. This will aid the development of comprehensive and relevant frameworks with which to fairly and transparently distribute research funds or approve promotion applications

From “a fair game” to “a form of covert research”:Research Ethics Committee Members differing notions of consent and potential risk to participants within social media research.

Social media (SM) research presents new challenges for research ethics committees (RECs) who must balance familiar ethical principles with new notions of public availability. This article qualitatively examines how U.K. REC members view this balance in terms of risk and consent. While it found significant variance overall, there were discernible experience-based trends. REC members with less experience of reviewing SM held inflexible notions of consent and risk that could be categorized as either relying on traditional notions of requiring direct consent, or viewing publicly available data as “fair game.” More experienced REC members took a more nuanced approach to data use and consent. We conclude that the more nuanced approach should be best practice during ethical review of SM research

Funding Infectious Disease Research: A Systematic Analysis of UK Research Investments by Funders 1997–2010

Background: Research investments are essential to address the burden of disease, however allocation of limited resources is poorly documented. We systematically reviewed the investments awarded by funding organisations to UK institutions and their global partners for infectious disease research. Methodology/Principal Findings Public and philanthropic investments for the period 1997 to 2010 were included. We categorised studies by infectious disease, cross-cutting theme, and by research and development value chain, reflecting the type of science. We identified 6165 funded studies, with a total research investment of UK £2.6 billion. Public organisations provided £1.4 billion (54.0%) of investments compared with £1.1 billion (42.4%) by philanthropic organisations. Global health studies represented an investment of £928 million (35.7%). The Wellcome Trust was the leading investor with £688 million (26.5%), closely followed by the UK Medical Research Council (MRC) with £673 million (25.9%). Funding over time was volatile, ranging from ∼£40 million to ∼£160 million per year for philanthropic organisations and ∼£30 million to ∼£230 million for public funders. Conclusions/Significance: Infectious disease research funding requires global coordination and strategic long-term vision. Our analysis demonstrates the diversity and inconsistent patterns in investment, with volatility in annual funding amounts and limited investment for product development and clinical trials

Worldwide topology of the scientific subject profile: a macro approach in the country level

[Background] Models for the production of knowledge and systems of innovation and science are key elements for characterizing a country in view of its scientific thematic profile. With regard to scientific output and publication in journals of international visibility, the countries of the world may be classified into three main groups according to their thematic bias.[Methodology/Principal Findings] This paper aims to classify the countries of the world in several broad groups, described in terms of behavioural models that attempt to sum up the characteristics of their systems of knowledge and innovation. We perceive three clusters in our analysis: 1) the biomedical cluster, 2) the basic science & engineering cluster, and 3) the agricultural cluster. The countries are conceptually associated with the clusters via Principal Component Analysis (PCA), and a Multidimensional Scaling (MDS) map with all the countries is presented.[Conclusions/Significance] As we have seen, insofar as scientific output and publication in journals of international visibility is concerned, the countries of the world may be classified into three main groups according to their thematic profile. These groups can be described in terms of behavioral models that attempt to sum up the characteristics of their systems of knowledge and innovation

Peer review quality and transparency of the peer-review process in open access and subscription journals

BACKGROUND:Recent controversies highlighting substandard peer review in Open Access (OA) and traditional (subscription) journals have increased the need for authors, funders, publishers, and institutions to assure quality of peer-review in academic journals. I propose that transparency of the peer-review process may be seen as an indicator of the quality of peer-review, and develop and validate a tool enabling different stakeholders to assess transparency of the peer-review process. METHODS AND FINDINGS:Based on editorial guidelines and best practices, I developed a 14-item tool to rate transparency of the peer-review process on the basis of journals' websites. In Study 1, a random sample of 231 authors of papers in 92 subscription journals in different fields rated transparency of the journals that published their work. Authors' ratings of the transparency were positively associated with quality of the peer-review process but unrelated to journal's impact factors. In Study 2, 20 experts on OA publishing assessed the transparency of established (non-OA) journals, OA journals categorized as being published by potential predatory publishers, and journals from the Directory of Open Access Journals (DOAJ). Results show high reliability across items (α = .91) and sufficient reliability across raters. Ratings differentiated the three types of journals well. In Study 3, academic librarians rated a random sample of 140 DOAJ journals and another 54 journals that had received a hoax paper written by Bohannon to test peer-review quality. Journals with higher transparency ratings were less likely to accept the flawed paper and showed higher impact as measured by the h5 index from Google Scholar. CONCLUSIONS:The tool to assess transparency of the peer-review process at academic journals shows promising reliability and validity. The transparency of the peer-review process can be seen as an indicator of peer-review quality allowing the tool to be used to predict academic quality in new journals

How Can the Operating Environment for Nutrition Research Be Improved in Sub-Saharan Africa? The Views of African Researchers

Optimal nutrition is critical for human development and economic growth. Sub-Saharan Africa is facing high levels of food insecurity and only few sub-Saharan African countries are on track to eradicate extreme poverty and hunger by 2015. Effective research capacity is crucial for addressing emerging challenges and designing appropriate mitigation strategies in sub-Saharan Africa. A clear understanding of the operating environment for nutrition research in sub-Saharan Africa is a much needed prerequisite. We collected data on the barriers and requirements for conducting nutrition research in sub-Saharan Africa through semi-structured interviews with 144 participants involved in nutrition research in 35 countries in sub-Saharan Africa. A total of 133 interviews were retained for coding. The main barriers identified for effective nutrition research were the lack of funding due to poor recognition by policymakers of the importance of nutrition research and under-utilisation of research findings for developing policy, as well as an absence of research priority setting from within Africa. Current research topics were perceived to be mainly determined by funding bodies from outside Africa. Nutrition researchers argued for more commitment from policymakers at national level. The low capacity for nutrition research was mainly seen as a consequence of insufficient numbers of nutrition researchers, limited skills and a poor research infrastructure. In conclusion, African nutrition researchers argued how research priorities need to be identified by African stakeholders, accompanied by consensus building to enable creating a problem-driven national research agenda. In addition, it was considered necessary to promote interactions among researchers, and between researchers and policymakers. Multidisciplinary research and international and cross-African collaboration were seen as crucial to build capacity in sub-Saharan nutrition research

How does the tobacco industry attempt to influence marketing regulations? A systematic review

BACKGROUND: The Framework Convention on Tobacco Control makes a number of recommendations aimed at restricting the marketing of tobacco products. Tobacco industry political activity has been identified as an obstacle to Parties' development and implementation of these provisions. This study systematically reviews the existing literature on tobacco industry efforts to influence marketing regulations and develops taxonomies of 1) industry strategies and tactics and 2) industry frames and arguments. METHODS: Searches were conducted between April-July 2011, and updated in March 2013. Articles were included if they made reference to tobacco industry efforts to influence marketing regulations; supported claims with verifiable evidence; were written in English; and concerned the period 1990-2013. 48 articles met the review criteria. Narrative synthesis was used to combine the evidence. RESULTS: 56% of articles focused on activity in North America, Europe or Australasia, the rest focusing on Asia (17%), South America, Africa or transnational activity. Six main political strategies and four main frames were identified. The tobacco industry frequently claims that the proposed policy will have negative unintended consequences, that there are legal barriers to regulation, and that the regulation is unnecessary because, for example, industry does not market to youth or adheres to a voluntary code. The industry primarily conveys these arguments through direct and indirect lobbying, the promotion of voluntary codes and alternative policies, and the formation of alliances with other industrial sectors. The majority of tactics and arguments were used in multiple jurisdictions. CONCLUSIONS: Tobacco industry political activity is far more diverse than suggested by existing taxonomies of corporate political activity. Tactics and arguments are repeated across jurisdictions, suggesting that the taxonomies of industry tactics and arguments developed in this paper are generalisable to multiple jurisdictions and can be used to predict industry activity

BHPR research: qualitative1. Complex reasoning determines patients' perception of outcome following foot surgery in rheumatoid arhtritis

Background: Foot surgery is common in patients with RA but research into surgical outcomes is limited and conceptually flawed as current outcome measures lack face validity: to date no one has asked patients what is important to them. This study aimed to determine which factors are important to patients when evaluating the success of foot surgery in RA Methods: Semi structured interviews of RA patients who had undergone foot surgery were conducted and transcribed verbatim. Thematic analysis of interviews was conducted to explore issues that were important to patients. Results: 11 RA patients (9 ♂, mean age 59, dis dur = 22yrs, mean of 3 yrs post op) with mixed experiences of foot surgery were interviewed. Patients interpreted outcome in respect to a multitude of factors, frequently positive change in one aspect contrasted with negative opinions about another. Overall, four major themes emerged. Function: Functional ability & participation in valued activities were very important to patients. Walking ability was a key concern but patients interpreted levels of activity in light of other aspects of their disease, reflecting on change in functional ability more than overall level. Positive feelings of improved mobility were often moderated by negative self perception ("I mean, I still walk like a waddling duck”). Appearance: Appearance was important to almost all patients but perhaps the most complex theme of all. Physical appearance, foot shape, and footwear were closely interlinked, yet patients saw these as distinct separate concepts. Patients need to legitimize these feelings was clear and they frequently entered into a defensive repertoire ("it's not cosmetic surgery; it's something that's more important than that, you know?”). Clinician opinion: Surgeons' post operative evaluation of the procedure was very influential. The impact of this appraisal continued to affect patients' lasting impression irrespective of how the outcome compared to their initial goals ("when he'd done it ... he said that hasn't worked as good as he'd wanted to ... but the pain has gone”). Pain: Whilst pain was important to almost all patients, it appeared to be less important than the other themes. Pain was predominately raised when it influenced other themes, such as function; many still felt the need to legitimize their foot pain in order for health professionals to take it seriously ("in the end I went to my GP because it had happened a few times and I went to an orthopaedic surgeon who was quite dismissive of it, it was like what are you complaining about”). Conclusions: Patients interpret the outcome of foot surgery using a multitude of interrelated factors, particularly functional ability, appearance and surgeons' appraisal of the procedure. While pain was often noted, this appeared less important than other factors in the overall outcome of the surgery. Future research into foot surgery should incorporate the complexity of how patients determine their outcome Disclosure statement: All authors have declared no conflicts of interes

Evaluating the Effects of SARS-CoV-2 Spike Mutation D614G on Transmissibility and Pathogenicity.

Global dispersal and increasing frequency of the SARS-CoV-2 spike protein variant D614G are suggestive of a selective advantage but may also be due to a random founder effect. We investigate the hypothesis for positive selection of spike D614G in the United Kingdom using more than 25,000 whole genome SARS-CoV-2 sequences. Despite the availability of a large dataset, well represented by both spike 614 variants, not all approaches showed a conclusive signal of positive selection. Population genetic analysis indicates that 614G increases in frequency relative to 614D in a manner consistent with a selective advantage. We do not find any indication that patients infected with the spike 614G variant have higher COVID-19 mortality or clinical severity, but 614G is associated with higher viral load and younger age of patients. Significant differences in growth and size of 614G phylogenetic clusters indicate a need for continued study of this variant