Understanding the relative valuation of research impact: a best-worst scaling experiment of the general public and biomedical and health researchers

Objectives: (1) To test the use of Best Worst Scaling (BWS) experiments in valuing different types of biomedical and health research impact, and (2) to explore how different types of research impact are valued by different stakeholder groups. Design: Survey-based BWS experiment and discrete choice modelling. Setting: United Kingdom. Participants: Current and recent UK Medical Research Council grant holders and a representative sample of the general public recruited from an online panel. Results: In relation to the study’s two objectives: (1) We demonstrate the application of BWS methodology in the quantitative assessment and valuation of research impact. (2) The general public and researchers provided similar valuations for research impacts such as improved life expectancy, job creation and reduced health costs, but there was less agreement between the groups on other impacts, including commercial capacity development, training and dissemination. Conclusion: This is the second time that a discrete choice experiment has been used to assess how the general public and researchers value different types of research impact, and the first time that BWS has been used to elicit these choices. While the two groups value different research impacts in different ways, we note that where they agree, this is generally about matters that are seemingly more important and associated with wider social benefit, rather than impacts occurring within the research system. These findings are a first step in exploring how the beneficiaries and producers of research value different kinds of impact, an important consideration given the growing emphasis on funding and assessing research on the basis of (potential) impact. Future research should refine and replicate both the current study and that of Miller et al. (2013) in other countries and disciplines. Strength

Understanding factors associated with the translation of cardiovascular research: A multinational case study approach

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited.This article has been made available through the Brunel Open Access Publishing Fund.Background: Funders of health research increasingly seek to understand how best to allocate resources in order to achieve maximum value from their funding. We built an international consortium and developed a multinational case study approach to assess benefits arising from health research. We used that to facilitate analysis of factors in the production of research that might be associated with translating research findings into wider impacts, and the complexities involved. Methods: We built on the Payback Framework and expanded its application through conducting co-ordinated case studies on the payback from cardiovascular and stroke research in Australia, Canada and the United Kingdom. We selected a stratified random sample of projects from leading medical research funders. We devised a series of innovative steps to: minimize the effect of researcher bias; rate the level of impacts identified in the case studies; and interrogate case study narratives to identify factors that correlated with achieving high or low levels of impact. Results: Twenty-nine detailed case studies produced many and diverse impacts. Over the 15 to 20 years examined, basic biomedical research has a greater impact than clinical research in terms of academic impacts such as knowledge production and research capacity building. Clinical research has greater levels of wider impact on health policies, practice, and generating health gains. There was no correlation between knowledge production and wider impacts. We identified various factors associated with high impact. Interaction between researchers and practitioners and the public is associated with achieving high academic impact and translation into wider impacts, as is basic research conducted with a clinical focus. Strategic thinking by clinical researchers, in terms of thinking through pathways by which research could potentially be translated into practice, is associated with high wider impact. Finally, we identified the complexity of factors behind research translation that can arise in a single case. Conclusions: We can systematically assess research impacts and use the findings to promote translation. Research funders can justify funding research of diverse types, but they should not assume academic impacts are proxies for wider impacts. They should encourage researchers to consider pathways towards impact and engage potential research users in research processes. © 2014 Wooding et al.; licensee BioMed Central Ltd.RAND Europe and HERG, with subsequent funding from the NHFA, the HSFC and the CIHR. This research was also partially supported by the Policy Research Programme in the English Department of Health

Estimating the returns to UK publicly funded cancer-related research in terms of the net value of improved health outcomes

© 2014 Glover et al.; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.Background - Building on an approach developed to assess the economic returns to cardiovascular research, we estimated the economic returns from UK public and charitable funded cancer-related research that arise from the net value of the improved health outcomes. Methods - To assess these economic returns from cancer-related research in the UK we estimated: 1) public and charitable expenditure on cancer-related research in the UK from 1970 to 2009; 2) net monetary benefit (NMB), that is, the health benefit measured in quality adjusted life years (QALYs) valued in monetary terms (using a base-case value of a QALY of GB£25,000) minus the cost of delivering that benefit, for a prioritised list of interventions from 1991 to 2010; 3) the proportion of NMB attributable to UK research; 4) the elapsed time between research funding and health gain; and 5) the internal rate of return (IRR) from cancer-related research investments on health benefits. We analysed the uncertainties in the IRR estimate using sensitivity analyses to illustrate the effect of some key parameters. Results - In 2011/12 prices, total expenditure on cancer-related research from 1970 to 2009 was £15 billion. The NMB of the 5.9 million QALYs gained from the prioritised interventions from 1991 to 2010 was £124 billion. Calculation of the IRR incorporated an estimated elapsed time of 15 years. We related 17% of the annual NMB estimated to be attributable to UK research (for each of the 20 years 1991 to 2010) to 20 years of research investment 15 years earlier (that is, for 1976 to 1995). This produced a best-estimate IRR of 10%, compared with 9% previously estimated for cardiovascular disease research. The sensitivity analysis demonstrated the importance of smoking reduction as a major source of improved cancer-related health outcomes. Conclusions - We have demonstrated a substantive IRR from net health gain to public and charitable funding of cancer-related research in the UK, and further validated the approach that we originally used in assessing the returns from cardiovascular research. In doing so, we have highlighted a number of weaknesses and key assumptions that need strengthening in further investigations. Nevertheless, these cautious estimates demonstrate that the returns from past cancer research have been substantial, and justify the investments made during the period 1976 to 1995.Wellcome Trust, Cancer Research UK, the National Institute of Health Research, and the Academy of Medical Sciences

Robust estimation of bacterial cell count from optical density

Optical density (OD) is widely used to estimate the density of cells in liquid culture, but cannot be compared between instruments without a standardized calibration protocol and is challenging to relate to actual cell count. We address this with an interlaboratory study comparing three simple, low-cost, and highly accessible OD calibration protocols across 244 laboratories, applied to eight strains of constitutive GFP-expressing E. coli. Based on our results, we recommend calibrating OD to estimated cell count using serial dilution of silica microspheres, which produces highly precise calibration (95.5% of residuals <1.2-fold), is easily assessed for quality control, also assesses instrument effective linear range, and can be combined with fluorescence calibration to obtain units of Molecules of Equivalent Fluorescein (MEFL) per cell, allowing direct comparison and data fusion with flow cytometry measurements: in our study, fluorescence per cell measurements showed only a 1.07-fold mean difference between plate reader and flow cytometry data

Estimating the Returns to United Kingdom Publicly Funded Musculoskeletal Disease Research in Terms of Net Value of Improved Health Outcomes

Abstract Background Building on an approach applied to cardiovascular and cancer research, we estimated the economic returns from United Kingdom public- and charitable-funded musculoskeletal disease (MSD) research that arise from the net value of the improved health outcomes in the United Kingdom. Methods To calculate the economic returns from MSD-related research in the United Kingdom, we estimated (1) the public and charitable expenditure on MSD-related research in the United Kingdom between 1970 and 2013; (2) the net monetary benefit (NMB), derived from the health benefit in quality adjusted life years (QALYs) valued in monetary terms (using a base-case value of a QALY of £25,000) minus the cost of delivering that benefit, for a prioritised list of interventions from 1994 to 2013; (3) the proportion of NMB attributable to United Kingdom research; and (4) the elapsed time between research funding and health gain. The data collected from these four key elements were used to estimate the internal rate of return (IRR) from MSD-related research investments on health benefits. We analysed the uncertainties in the IRR estimate using a one-way sensitivity analysis. Results Expressed in 2013 prices, total expenditure on MSD-related research from 1970 to 2013 was £3.5 billion, and for the period used to estimate the rate of return, 1978-1997, was £1.4 billion. Over the period 1994–2013 the key interventions analysed produced 871,000 QALYs with a NMB of £16 billion, allowing for the net NHS costs resulting from them and valuing a QALY at £25,000. The proportion of benefit attributable to United Kingdom research was 30% and the elapsed time between funding and impact of MSD treatments was 16 years. Our best estimate of the IRR from MSD-related research was 7%, which is similar to the 9% for CVD and 10% for cancer research. Conclusions Our estimate of the IRR from the net health gain to public and charitable funding of MSD-related research in the United Kingdom is substantial, and justifies the research investments made between 1978 and 1997. We also demonstrated the applicability of the approach previously used in assessing the returns from cardiovascular and cancer research. Inevitably, with a study of this kind, there are a number of important assumptions and caveats that we highlight, and these can inform future research

Does a biomedical research centre affect patient care in local hospitals?

Abstract Background Biomedical research can have impacts on patient care at research-active hospitals. We qualitatively evaluated the impact of the Oxford Biomedical Research Centre (Oxford BRC), a university-hospital partnership, on the effectiveness and efficiency of healthcare in local hospitals. Effectiveness and efficiency are conceptualised in terms of impacts perceived by clinicians on the quality, quantity and costs of patient care they deliver. Methods First, we reviewed documentation from Oxford BRC and literature on the impact of research activity on patient care. Second, we interviewed leaders of the Oxford BRC\u2019s research to identify the direct and indirect impacts they expected their activity would have on local hospitals. Third, this information was used to inform interviews with senior clinicians responsible for patient care at Oxford\u2019s acute hospitals to discover what impacts they observed from research generally and from Oxford BRC\u2019s research work specifically. We compared and contrasted the results from the two sets of interviews using a qualitative approach. Finally, we identified themes emerging from the senior clinicians\u2019 responses, and compared them with an existing taxonomy of mechanisms through which quality of healthcare may be affected in research-active settings. Results We were able to interview 17 research leaders at the Oxford BRC and 19 senior clinicians at Oxford\u2019s acute hospitals. The research leaders identified a wide range of beneficial impacts that they expected might be felt at local hospitals as a result of their research activity. They expected the impact of their research activity on patient care to be generally positive. The senior clinicians responsible for patient care at those hospitals presented a more mixed picture, identifying many positive impacts, but also a smaller number of negative impacts, from research activity, including that of the Oxford BRC. We found the existing taxonomy of benefit types to be helpful in organising the findings, and propose modifications to further improve its usefulness. Conclusions Impacts from research activity on the effectiveness and efficiency of patient care at the local acute hospitals, as perceived by senior clinicians, were more often beneficial than harmful. The Oxford BRC contributed to those impacts

Estimating the returns to UK publicly funded cancer-related research in terms of the net value of improved health outcomes

Background Building on an approach developed to assess the economic returns to cardiovascular research, we estimated the economic returns from UK public and charitable funded cancer-related research that arise from the net value of the improved health outcomes. Methods To assess these economic returns from cancer-related research in the UK we estimated: 1) public and charitable expenditure on cancer-related research in the UK from 1970 to 2009; 2) net monetary benefit (NMB), that is, the health benefit measured in quality adjusted life years (QALYs) valued in monetary terms (using a base-case value of a QALY of GB£25,000) minus the cost of delivering that benefit, for a prioritised list of interventions from 1991 to 2010; 3) the proportion of NMB attributable to UK research; 4) the elapsed time between research funding and health gain; and 5) the internal rate of return (IRR) from cancer-related research investments on health benefits. We analysed the uncertainties in the IRR estimate using sensitivity analyses to illustrate the effect of some key parameters. Results In 2011/12 prices, total expenditure on cancer-related research from 1970 to 2009 was £15 billion. The NMB of the 5.9 million QALYs gained from the prioritised interventions from 1991 to 2010 was £124 billion. Calculation of the IRR incorporated an estimated elapsed time of 15 years. We related 17% of the annual NMB estimated to be attributable to UK research (for each of the 20 years 1991 to 2010) to 20 years of research investment 15 years earlier (that is, for 1976 to 1995). This produced a best-estimate IRR of 10%, compared with 9% previously estimated for cardiovascular disease research. The sensitivity analysis demonstrated the importance of smoking reduction as a major source of improved cancer-related health outcomes. Conclusions We have demonstrated a substantive IRR from net health gain to public and charitable funding of cancer-related research in the UK, and further validated the approach that we originally used in assessing the returns from cardiovascular research. In doing so, we have highlighted a number of weaknesses and key assumptions that need strengthening in further investigations. Nevertheless, these cautious estimates demonstrate that the returns from past cancer research have been substantial, and justify the investments made during the period 1976 to 1995.</p