Conducting retrospective impact analysis to inform a medical research charity’s funding strategies: The case of Asthma UK

© 2013 Hanney et al.; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.This article has been made available through the Brunel Open Access Publishing Fund.BACKGROUND: Debate is intensifying about how to assess the full range of impacts from medical research. Complexity increases when assessing the diverse funding streams of funders such as Asthma UK, a charitable patient organisation supporting medical research to benefit people with asthma. This paper aims to describe the various impacts identified from a range of Asthma UK research, and explore how Asthma UK utilised the characteristics of successful funding approaches to inform future research strategies. METHODS: We adapted the Payback Framework, using it both in a survey and to help structure interviews, documentary analysis, and case studies. We sent surveys to 153 lead researchers of projects, plus 10 past research fellows, and also conducted 14 detailed case studies. These covered nine projects and two fellowships, in addition to the innovative case studies on the professorial chairs (funded since 1988) and the MRC-Asthma UK Centre in Allergic Mechanisms of Asthma (the ‘Centre’) which together facilitated a comprehensive analysis of the whole funding portfolio. We organised each case study to capture whatever academic and wider societal impacts (or payback) might have arisen given the diverse timescales, size of funding involved, and extent to which Asthma UK funding contributed to the impacts. RESULTS: Projects recorded an average of four peer-reviewed journal articles. Together the chairs reported over 500 papers. All streams of funding attracted follow-on funding. Each of the various categories of societal impacts arose from only a minority of individual projects and fellowships. Some of the research portfolio is influencing asthma-related clinical guidelines, and some contributing to product development. The latter includes potentially major breakthroughs in asthma therapies (in immunotherapy, and new inhaled drugs) trialled by university spin-out companies. Such research-informed guidelines and medicines can, in turn, contribute to health improvements. The role of the chairs and the pioneering collaborative Centre is shown as being particularly important. CONCLUSIONS: We systematically demonstrate that all types of Asthma UK’s research funding assessed are making impacts at different levels, but the main societal impacts from projects and fellowships come from a minority of those funded. Asthma UK used the study’s findings, especially in relation to the Centre, to inform research funding strategies to promote the achievement of impact.This study was funded by Asthma UK

An evidence base to optimise methods for involving patient and public contributors in clinical trials: a mixed-methods study

BACKGROUND: In comparison with other study designs, randomised trials are regarded as particularly likely to benefit from patient and public involvement (PPI). Using mixed-methods research we investigated PPI from the perspectives of researchers and PPI contributors. METHODS: Randomised trials in receipt of funding from the Health Technology Assessment (HTA) programme between 2006 and 2010 were identified. Funding applications and board and referee comments were obtained and data relevant to PPI extracted. Chief investigators (CIs), PPI contributors and UK Clinical Research Collaboration Registered Clinical Trials Units (RCTUs) were surveyed. Interviews were conducted with researchers and PPI contributors. RESULTS: A total of 111 trials were included. Text relevant to PPI was identified in half of the trials for which the first-stage applications were available, but only one-quarter described PPI within their development. In the second stage of the application, the majority provided some text relevant to PPI, with over half having PPI in their development. Fewer than half of referees commented on PPI, and funding boards rarely provided comments in relation to PPI. Seventy-three per cent (81 of 111) of CIs responded to the survey and 98% (79 of 81) included PPI at some stage in their trial. CIs considered high impact from PPI contributors to occur more frequently in trial setup, with low or no impact being more common during trial conduct, analysis and dissemination. Only one-third of CIs provided PPI contributor contact details but all contributors contacted completed the survey. The majority of contributors felt engaged and valued by the research team. Interviews were conducted with researchers and/or PPI contributors for 28 trials identifying two main influences on perception of PPI impact: whether or not CIs expressed personal goals and plans for PPI; and the quality of their relationship with the PPI contributors. The importance of early engagement was identified, with opportunity for input thereafter limited. Three PPI roles were identified: oversight, managerial and responsive. Oversight roles, as required by funders, were associated with low impact in comparison with responsive or managerial roles. Most researchers could see some value in PPI training for researchers, although those that had received such training themselves expressed concerns about its purpose and evidence base. Training for PPI contributors was considered unnecessary, with conversational approaches preferred, although this did not appear to provide an opportunity for role negotiation. The RCTU survey response rate was 85% (39 of 46). The majority (37 of 39) reported PPI within trials co-ordinated by their unit. Trial characteristics were used by half to determine the approach to PPI. Two-thirds reported recent developments or changes in implementing plans for PPI (21 of 33). Support to PPI contributors was commonly offered through members of staff at the unit. CONCLUSIONS: PPI is occurring in the majority of trials funded by the HTA programme, but uncertainty remains about how it is assessed and valued. Early involvement, building a relationship between researchers and contributors, responsive or managerial roles, and having defined goals for PPI were associated with impact. Efficiency could be gained by utilising the RCTU network to identify and tackle challenges, and develop a risk-based approach utilising trial characteristics. Recommendations are made to trial funders and the research community. Given the difficulties for some informants in recalling PPI contributions, future research using a prospective approach would be valuable. Ethnographic research that combines observation and multi-informant interviews is likely to be informative in identifying impact. The research community needs to give further consideration to processes for selecting PPI contributors and models of implementing PPI

Conceptual frameworks and empirical approaches used to assess the impact of health research: an overview of reviews

<p>Abstract</p> <p>Background</p> <p>How to assess the impact of research is of growing interest to funders, policy makers and researchers mainly to understand the value of investments and to increase accountability. Broadly speaking the term "research impact" refers to the contribution of research activities to achieve desired societal outcomes. The aim of this overview is to identify the most common approaches to research impact assessment, categories of impact and their respective indicators.</p> <p>Methods</p> <p>We systematically searched the relevant literature (PubMed, The Cochrane Library (1990-2009)) and funding agency websites. We included systematic reviews, theoretical and methodological papers, and empirical case-studies on how to evaluate research impact. We qualitatively summarised the included reports, as well the conceptual frameworks.</p> <p>Results</p> <p>We identified twenty-two reports belonging to four systematic reviews and 14 primary studies. These publications reported several theoretical frameworks and methodological approaches (bibliometrics, econometrics, ad hoc case studies). The "payback model" emerged as the most frequently used. Five broad categories of impact were identified: a) advancing knowledge, b) capacity building, c) informing decision-making, d) health benefits, e) broad socio-economic benefits. For each proposed category of impact we summarized a set of indicators whose pros and cons are presented and briefly discussed.</p> <p>Conclusions</p> <p>This overview is a comprehensive, yet descriptive, contribution to summarize the conceptual framework and taxonomy of an heterogeneous and evolving area of research. A shared and comprehensive conceptual framework does not seem to be available yet and its single components (epidemiologic, economic, and social) are often valued differently in different models.</p

Research utilisation and knowledge mobilisation in the commissioning and joint planning of public health interventions to reduce alcohol-related harms: a qualitative case design using a cocreation approach

Background: Considerable resources are spent on research to establish what works to improve the nation’s health. If the findings from this research are used, better health outcomes can follow, but we know that these findings are not always used. In public health, evidence of what works may not ‘fit’ everywhere, making it difficult to know what to do locally. Research suggests that evidence use is a social and dynamic process, not a simple application of research findings. It is unclear whether it is easier to get evidence used via a legal contracting process or within unified organisational arrangements with shared responsibilities. Objective: To work in cocreation with research participants to investigate how research is utilised and knowledge mobilised in the commissioning and planning of public health services to reduce alcohol-related harms. Design, setting and participants: Two in-depth, largely qualitative, cross-comparison case studies were undertaken to compare real-time research utilisation in commissioning across a purchaser–provider split (England) and in joint planning under unified organisational arrangements (Scotland) to reduce alcohol-related harms. Using an overarching realist approach and working in cocreation, case study partners (stakeholders in the process) picked the topic and helped to interpret the findings. In Scotland, the topic picked was licensing; in England, it was reducing maternal alcohol consumption. Methods: Sixty-nine interviews, two focus groups, 14 observations of decision-making meetings, two local feedback workshops (n = 23 and n = 15) and one national workshop (n = 10) were undertaken. A questionnaire (n = 73) using a Behaviourally Anchored Rating Scale was issued to test the transferability of the 10 main findings. Given the small numbers, care must be taken in interpreting the findings. Findings: Not all practitioners have the time, skills or interest to work in cocreation, but when there was collaboration, much was learned. Evidence included professional and tacit knowledge, and anecdotes, as well as findings from rigorous research designs. It was difficult to identify evidence in use and decisions were sometimes progressed in informal ways and in places we did not get to see. There are few formal evidence entry points. Evidence (prevalence and trends in public health issues) enters the process and is embedded in strategic documents to set priorities, but local data were collected in both sites to provide actionable messages (sometimes replicating the evidence base). Conclusions: Two mid-range theories explain the findings. If evidence has saliency (relates to ‘here and now’ as opposed to ‘there and then’) and immediacy (short, presented verbally or visually and with emotional appeal) it is more likely to be used in both settings. A second mid-range theory explains how differing tensions pull and compete as feasible and acceptable local solutions are pursued across stakeholders. Answering what works depends on answering for whom and where simultaneously to find workable (if temporary) ‘blends’. Gaining this agreement across stakeholders appeared more difficult across the purchaser–provider split, because opportunities to interact were curtailed; however, more research is needed. Funding: This study was funded by the Health Services and Delivery Research programme of the National Institute for Health Research

Circumcision: An Informed Choice

Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/73534/1/1091592305277052.pd