A systems approach for optimizing implementation to impact: meeting report and proceedings of the 2019 In the Trenches: Implementation to Impact International Summit

Background The In the Trenches series of cutting-edge knowledge sharing events on impact for front-line experts and practitioners provides an engagement platform for diverse stakeholders across government, research funding organizations, industry, and academia to share emerging knowledge and practical experiences. The second event of the series In the Trenches: Implementation to Impact International Summit was held in Banff, Alberta, Canada, on June 7–8, 2019. The overarching vision for the Summit was to create an engagement platform for addressing key challenges and finding practical solutions to move from implementation (i.e. putting findings into effect) to impact (i.e. creating benefits to society and the economy). Processes and proceedings The Summit used diverse approaches to facilitate active engagement and knowledge sharing between 80 delegates across sectors and jurisdictions. Summit sessions mostly consisted of short talks and moderated panels grouped into eight thematic sessions. Each presentation included a summary of Key Messages, along with a summary of the Actionable Insights which concluded each session. The presentations and discussions are analysed, synthesized and described in this proceedings paper using a systems approach. This demonstrates how the Summit focused on each of the necessary functions (and associated components) that should be undertaken, and combined, for effective research and innovation: stewardship and governance, securing finance, creating capacity, and producing and using research. The approach also identifies relevant challenges. Conclusions There is increased interest globally in the benefits that can accrue from adopting a systems approach to research and innovation. Various organizations in Canada and internationally have made considerable progress on Implementation to Impact, often as a result of well-planned initiatives. The Summit highlights the value of 1) collaboration between researchers and potential users, and 2) the adoption by funders of approaches involving an increasing range of responsibilities and activities. The Summit website (https://inthetrenchessummit.com/) will be periodically updated with new resources and information about future In the Trenches events.Alberta Innovates; European Union’s Horizon 2020 research and innovation programme award STARBIOS2 (grant agreement No. 709517); National Institute for Health Research (grant NIHR Oxford Biomedical Research Centre (BRC-1215-20008) to the Oxford University Hospitals NHS Foundation Trust and the University of Oxford)

The impact generated by public and charity-funded research in the UK: A systematic literature review

Objective: To identify, synthesize and critically assess the empirical evidence of the impact generated by public and charity funded health research in the United Kingdom. Methods: We conducted a systematic literature review of the empirical evidence published in English in peer-reviewed journals between 2006 and 2017. Studies meeting the inclusion criteria were selected and their findings were analysed using the Payback Framework into five main categories: knowledge, benefits to future research and research use, benefits from informing policy and product development, health and health sector benefits and broader economic benefits. We assessed the studies for risk of selection, reporting and funding bias. Results: Thirteen studies met the inclusion criteria. The majority of the studies (10 out of 13) assessed impact at multiple domains including the main 5 key themes of the Payback Framework. All of them showed a positive impact of funded research on outcomes. Of those studies, one presented low risk of bias (8%), 6 studies were classified as presenting moderate risk of bias (46%) and 6 studies presented high risk of bias (46%). Conclusions: Empirical evidence on the impact of public and charity funded research is still limited and subject to funding and selection bias. More work is needed to establish the causal effects of funded research on academic outcomes, policy, practice and the broader economy

Changing children’s understanding of the brain: a longitudinal study of the Royal Institution Christmas Lectures as a measure of public engagement

Demonstrating the impact of public engagement is an increasingly important activity for today’s academics and researchers. The difficulty is that many areas of interest do not lend themselves well to evaluation because the impact of each single intervention can be hard to trace and take time to become manifest. With this in mind, we evaluated a lecture based around the 2011 Royal Institution Christmas Lectures, ”Meet Your Brain,” delivered to school children from low performing schools. We compared knowledge about four neuroscience facts one week before, one week after and six weeks after the lecture. Analysis revealed significant knowledge transfer one week after the lecture that was retained five weeks later. We conclude that public engagement through tailored lectures can have significant impact in the moderate term with the potential to leave a lasting impression over a longer period

Inclusion of diverse populations in genomic research and health services: Genomix workshop report

Clinical genetic services and genomic research are rapidly developing but, historically, those with the greatest need are the least to benefit from these advances. This encompasses low-income communities, including those from ethnic minority and indigenous backgrounds. The “Genomix” workshop at the European Society of Human Genetics (ESHG) 2016 conference offered the opportunity to consider possible solutions for these disparities from the experiences of researchers and genetic healthcare practitioners working with underserved communities in the USA, UK and Australia. Evident from the workshop and corresponding literature is that a multi-faceted approach to engaging communities is essential. This needs to be complemented by redesigning healthcare systems that improves access and raises awareness of the needs of these communities. At a more strategic level, institutions involved in funding research, commissioning and redesigning genetic health services also need to be adequately represented by underserved populations with intrinsic mechanisms to disseminate good practice and monitor participation. Further, as genomic medicine is mainstreamed, educational programmes developed for clinicians should incorporate approaches to alleviate disparities in accessing genetic services and improving study participation

Building health research systems: WHO is generating global perspectives, and who’s celebrating national successes?

In 2016, England’s National Institute for Health Research (NIHR) celebrated its tenth anniversary as an innovative national health research system with a focus on meeting patients’ needs. This provides a good opportunity to reflect on how the creation of the NIHR has greatly enhanced important work, started in 1991, to develop a health research system in England that is embedded in the National Health Service. In 2004, WHO identified a range of functions that a national health research system should undertake to improve the health of populations. Health Research Policy and Systems (HRPS) has taken particular interest in the pioneering developments in the English health research system, where the comprehensive approach has covered most, if not all, of the functions identified by WHO. Furthermore, several significant recent developments in thinking about health research are relevant for the NIHR and have informed accounts of its achievements. These include recognition of the need to combat waste in health research, which had been identified as a global problem in successive papers in the Lancet, and an increasing emphasis on demonstrating impact. Here, pioneering evaluation of United Kingdom research, conducted through the impact case studies of the Research Excellence Framework, is particularly important. Analyses informed by these and other approaches identified many aspects of NIHR’s progress in combating waste, building and sustaining research capacity, creating centres of research excellence linked to leading healthcare institutions, developing research networks, involving patients and others in identifying research needs, and producing and adopting research findings that are improving health outcomes. The NIHR’s overall success, and an analysis of the remaining problems, might have lessons for other systems, notwithstanding important advances in many countries, as described in papers in HRPS and elsewhere. WHO’s recently established Global Observatory for Health Research and Development provides an opportunity to promote some of these lessons. To inform its work, the Observatory is sponsoring a thematic series of papers in HRPS focusing on health research issues such as funding flows, priority setting, capacity building, utilisation and equity. While important papers on these have been published, this series is still open to new submissions

Evaluation of the effects of implementing an electronic early warning score system: protocol for a stepped wedge study

Background: An Early Warning Score is a clinical risk score based upon vital signs intended to aid recognition of patients in need of urgent medical attention. The use of an escalation of care policy based upon an Early Warning Score is mandated as the standard of practice in British hospitals. Electronic systems for recording vital sign observations and Early Warning Score calculation offer theoretical benefits over paper-based systems. However, the evidence for their clinical benefit is limited. Previous studies have shown inconsistent results. The majority have employed a “before and after” study design, which may be strongly confounded by simultaneously occurring events. This study aims to examine how the implementation of an electronic early warning score system, System for Notification and Documentation (SEND), affects the recognition of clinical deterioration occurring in hospitalised adult patients. Methods: This study is a non-randomised stepped wedge evaluation carried out across the four hospitals of the Oxford University Hospitals NHS Trust, comparing charting on paper and charting using SEND. We assume that more frequent monitoring of acutely ill patients is associated with better recognition of patient deterioration. The primary outcome measure is the time between a patient’s first observations set with an Early Warning Score above the alerting threshold and their subsequent set of observations. Secondary outcome measures are in-hospital mortality, cardiac arrest and Intensive Care admission rates, hospital length of stay and system usability measured using the System Usability Scale. We will also measure Intensive Care length of stay, Intensive Care mortality, Acute Physiology and Chronic Health Evaluation (APACHE) II acute physiology score on admission, to examine whether the introduction of SEND has any effect on Intensive Care-related outcomes. Discussion: The development of this protocol has been informed by guidance from the Agency for Healthcare Research and Quality (AHRQ) Health Information Technology Evaluation Toolkit and Delone and McLeans’s Model of Information System Success. Our chosen trial design, a stepped wedge study, is well suited to the study of a phased roll out. The choice of primary endpoint is challenging. We have selected the time from the first triggering observation set to the subsequent observation set. This has the benefit of being easy to measure on both paper and electronic charting and having a straightforward interpretation. We have collected qualitative measures of system quality via a user questionnaire and organisational descriptors to help readers understand the context in which SEND has been implemented

Gendered health systems: evidence from low- and middle-income countries

Background Gender is often neglected in health systems, yet health systems are not gender neutral. Within health systems research, gender analysis seeks to understand how gender power relations create inequities in access to resources, the distribution of labour and roles, social norms and values, and decision-making. This paper synthesises findings from nine studies focusing on four health systems domains, namely human resources, service delivery, governance and financing. It provides examples of how a gendered and/or intersectional gender approach can be applied by researchers in a range of low- and middle-income settings (Cambodia, Zimbabwe, Uganda, India, China, Nigeria and Tanzania) to issues across the health system and demonstrates that these types of analysis can uncover new and novel ways of viewing seemingly intractable problems. Methods The research used a combination of mixed, quantitative, qualitative and participatory methods, demonstrating the applicability of diverse research methods for gender and intersectional analysis. Within each study, the researchers adapted and applied a variety of gender and intersectional tools to assist with data collection and analysis, including different gender frameworks. Some researchers used participatory tools, such as photovoice and life histories, to prompt deeper and more personal reflections on gender norms from respondents, whereas others used conventional qualitative methods (in-depth interviews, focus group discussion). Findings from across the studies were reviewed and key themes were extracted and summarised. Results Five core themes that cut across the different projects were identified and are reported in this paper as follows: the intersection of gender with other social stratifiers; the importance of male involvement; the influence of gendered social norms on health system structures and processes; reliance on (often female) unpaid carers within the health system; and the role of gender within policy and practice. These themes indicate the relevance of and need for gender analysis within health systems research. Conclusion The implications of the diverse examples of gender and health systems research highlighted indicate that policy-makers, health practitioners and others interested in enhancing health system research and delivery have solid grounds to advance their enquiry and that one-size-fits-all heath interventions that ignore gender and intersectionality dimensions require caution. It is essential that we build upon these insights in our efforts and commitment to move towards greater equity both locally and globally