Understanding the relative valuation of research impact: a best-worst scaling experiment of the general public and biomedical and health researchers

Objectives: (1) To test the use of Best Worst Scaling (BWS) experiments in valuing different types of biomedical and health research impact, and (2) to explore how different types of research impact are valued by different stakeholder groups. Design: Survey-based BWS experiment and discrete choice modelling. Setting: United Kingdom. Participants: Current and recent UK Medical Research Council grant holders and a representative sample of the general public recruited from an online panel. Results: In relation to the study’s two objectives: (1) We demonstrate the application of BWS methodology in the quantitative assessment and valuation of research impact. (2) The general public and researchers provided similar valuations for research impacts such as improved life expectancy, job creation and reduced health costs, but there was less agreement between the groups on other impacts, including commercial capacity development, training and dissemination. Conclusion: This is the second time that a discrete choice experiment has been used to assess how the general public and researchers value different types of research impact, and the first time that BWS has been used to elicit these choices. While the two groups value different research impacts in different ways, we note that where they agree, this is generally about matters that are seemingly more important and associated with wider social benefit, rather than impacts occurring within the research system. These findings are a first step in exploring how the beneficiaries and producers of research value different kinds of impact, an important consideration given the growing emphasis on funding and assessing research on the basis of (potential) impact. Future research should refine and replicate both the current study and that of Miller et al. (2013) in other countries and disciplines. Strength

Assessing the impact of health technology assessment in the Netherlands

Copyright © Cambridge University Press 2008Objectives: Investments in health research should lead to improvements in health and health care. This is also the remit of the main HTA program in the Netherlands. The aims of this study were to assess whether the results of this program have led to such improvements and to analyze how best to assess the impact from health research.Methods: We assessed the impact of individual HTA projects by adapting the "payback framework" developed in the United Kingdom. We conducted dossier reviews and sent a survey to principal investigators of forty-three projects awarded between 2000 and 2003. We then provided an overview of documented output and outcome that was assessed by ten HTA experts using a scoring method. Finally, we conducted five case studies using information from additional dossier review and semistructured key informant interviews.Results: The findings confirm that the payback framework is a useful approach to assess the impact of HTA projects. We identified over 101 peer reviewed papers, more than twenty-five PhDs, citations of research in guidelines (six projects), and implementation of new treatment strategies (eleven projects). The case studies provided greater depth and understanding about the levels of impact that arise and why and how they have been achieved.Conclusions: It is generally too early to determine whether the HTA program led to actual changes in healthcare policy and practice. However, the results can be used as a baseline measurement for future evaluation and can help funding organizations or HTA agencies consider how to assess impact, possibly routinely. This, in turn, could help inform research strategies and justify expenditure for health research.This research is funded by ZonMw, the Netherlands organization for health research and development (project 945-15-001)

From COVID-19 research to vaccine application: why might it take 17 months not 17 years and what are the wider lessons?

Abstract: It is often said that it takes 17 years to move medical research from bench to bedside. In a coronavirus disease (COVID-19) world, such time-lags feel intolerable. In these extraordinary circumstances could years be made into months? If so, could those lessons be used to accelerate medical research when the crisis eases? To measure time-lags in health and biomedical research as well as to identify ways of reducing them, we developed and published (in 2015) a matrix consisting of overlapping tracks (or stages/phases) in the translation from discovery research to developed products, policies and practice. The matrix aids analysis by highlighting the time and actions required to develop research (and its translation) both (1) along each track and (2) from one track to another, e.g. from the discovery track to the research-in-humans track. We noted four main approaches to reducing time-lags, namely increasing resources, working in parallel, starting or working at risk, and improving processes. Examining these approaches alongside the matrix helps interpret the enormous global effort to develop a vaccine for the 2019 novel coronavirus SARS-CoV-2, the causative agent of COVID-19. Rapid progress in the discovery/basic and human research tracks is being made through a combination of large-scale funding, work being conducted in parallel (between different teams globally and through working in overlapping tracks), working at greater (but proportionate) risk to safety than usual, and adopting various new processes. The overlapping work of some of the teams involves continuing animal research whilst entering vaccine candidates into Phase I trials alongside planning their Phase II trials. The additional funding available helps to reduce some of the usual financial risks in moving so quickly. Going forward through the increasingly large human trials for safety, dosage and efficacy, it will be vital to overlap work in parallel in the often challenging public policy and clinical tracks. Thus, regulatory and reimbursement bodies are beginning and preparing rapid action to pull vaccines proving to be safe and effective through to extraordinarily rapid application to the general population. Monitoring the development of a COVID-19 vaccine using the matrix (modified as necessary) could help identify which of the approaches speeding development and deployment could be usefully applied more widely in the future

From COVID-19 research to vaccine application: why might it take 17 months not 17 years and what are the wider lessons?

Abstract: It is often said that it takes 17 years to move medical research from bench to bedside. In a coronavirus disease (COVID-19) world, such time-lags feel intolerable. In these extraordinary circumstances could years be made into months? If so, could those lessons be used to accelerate medical research when the crisis eases? To measure time-lags in health and biomedical research as well as to identify ways of reducing them, we developed and published (in 2015) a matrix consisting of overlapping tracks (or stages/phases) in the translation from discovery research to developed products, policies and practice. The matrix aids analysis by highlighting the time and actions required to develop research (and its translation) both (1) along each track and (2) from one track to another, e.g. from the discovery track to the research-in-humans track. We noted four main approaches to reducing time-lags, namely increasing resources, working in parallel, starting or working at risk, and improving processes. Examining these approaches alongside the matrix helps interpret the enormous global effort to develop a vaccine for the 2019 novel coronavirus SARS-CoV-2, the causative agent of COVID-19. Rapid progress in the discovery/basic and human research tracks is being made through a combination of large-scale funding, work being conducted in parallel (between different teams globally and through working in overlapping tracks), working at greater (but proportionate) risk to safety than usual, and adopting various new processes. The overlapping work of some of the teams involves continuing animal research whilst entering vaccine candidates into Phase I trials alongside planning their Phase II trials. The additional funding available helps to reduce some of the usual financial risks in moving so quickly. Going forward through the increasingly large human trials for safety, dosage and efficacy, it will be vital to overlap work in parallel in the often challenging public policy and clinical tracks. Thus, regulatory and reimbursement bodies are beginning and preparing rapid action to pull vaccines proving to be safe and effective through to extraordinarily rapid application to the general population. Monitoring the development of a COVID-19 vaccine using the matrix (modified as necessary) could help identify which of the approaches speeding development and deployment could be usefully applied more widely in the future

Understanding factors associated with the translation of cardiovascular research: A multinational case study approach

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited.This article has been made available through the Brunel Open Access Publishing Fund.Background: Funders of health research increasingly seek to understand how best to allocate resources in order to achieve maximum value from their funding. We built an international consortium and developed a multinational case study approach to assess benefits arising from health research. We used that to facilitate analysis of factors in the production of research that might be associated with translating research findings into wider impacts, and the complexities involved. Methods: We built on the Payback Framework and expanded its application through conducting co-ordinated case studies on the payback from cardiovascular and stroke research in Australia, Canada and the United Kingdom. We selected a stratified random sample of projects from leading medical research funders. We devised a series of innovative steps to: minimize the effect of researcher bias; rate the level of impacts identified in the case studies; and interrogate case study narratives to identify factors that correlated with achieving high or low levels of impact. Results: Twenty-nine detailed case studies produced many and diverse impacts. Over the 15 to 20 years examined, basic biomedical research has a greater impact than clinical research in terms of academic impacts such as knowledge production and research capacity building. Clinical research has greater levels of wider impact on health policies, practice, and generating health gains. There was no correlation between knowledge production and wider impacts. We identified various factors associated with high impact. Interaction between researchers and practitioners and the public is associated with achieving high academic impact and translation into wider impacts, as is basic research conducted with a clinical focus. Strategic thinking by clinical researchers, in terms of thinking through pathways by which research could potentially be translated into practice, is associated with high wider impact. Finally, we identified the complexity of factors behind research translation that can arise in a single case. Conclusions: We can systematically assess research impacts and use the findings to promote translation. Research funders can justify funding research of diverse types, but they should not assume academic impacts are proxies for wider impacts. They should encourage researchers to consider pathways towards impact and engage potential research users in research processes. © 2014 Wooding et al.; licensee BioMed Central Ltd.RAND Europe and HERG, with subsequent funding from the NHFA, the HSFC and the CIHR. This research was also partially supported by the Policy Research Programme in the English Department of Health

Bitter Taste Receptors Influence Glucose Homeostasis

TAS1R- and TAS2R-type taste receptors are expressed in the gustatory system, where they detect sweet- and bitter-tasting stimuli, respectively. These receptors are also expressed in subsets of cells within the mammalian gastrointestinal tract, where they mediate nutrient assimilation and endocrine responses. For example, sweeteners stimulate taste receptors on the surface of gut enteroendocrine L cells to elicit an increase in intracellular Ca2+ and secretion of the incretin hormone glucagon-like peptide-1 (GLP-1), an important modulator of insulin biosynthesis and secretion. Because of the importance of taste receptors in the regulation of food intake and the alimentary responses to chemostimuli, we hypothesized that differences in taste receptor efficacy may impact glucose homeostasis. To address this issue, we initiated a candidate gene study within the Amish Family Diabetes Study and assessed the association of taste receptor variants with indicators of glucose dysregulation, including a diagnosis of type 2 diabetes mellitus and high levels of blood glucose and insulin during an oral glucose tolerance test. We report that a TAS2R haplotype is associated with altered glucose and insulin homeostasis. We also found that one SNP within this haplotype disrupts normal responses of a single receptor, TAS2R9, to its cognate ligands ofloxacin, procainamide and pirenzapine. Together, these findings suggest that a functionally compromised TAS2R receptor negatively impacts glucose homeostasis, providing an important link between alimentary chemosensation and metabolic disease

Surveying the scene - the RAISS tool for mapping the impact of research portfolios

Steven Wooding will describe the RAISS* tool - a low burden method of mapping the impacts of biomedical research across the research portfolio. The main body of RAISS is around 250 Yes/No questions that explore what impacts a piece of research has contributed to. By using high level questions to determine which detail questions are asked and by asking exclusively Yes/No questions the average completion time of the questionnaire is just over half an hour, with 90% of questionnaires completed in less than an hour. Despite this quick completion time the questionnaire still collects information across a huge range of impacts including on career development; collaboration within and outside academia; research capacity building, research tool production, dissemination in academic and non-academic contexts, impacts on health policy through a variety of routes; impacts on education and training of others and a wide variety of intervention, product development and public health advice outcomes. RAISS provides an overview of research impact and a basis for more detailed examination of particular research evaluation questions focusing on the 'why and how' of translation. We have also developed "Impact Arrays" as a way to represent the impacts of research and give an instant overview of the research portfolio and the ease of completion can allow tracking of impacts over time