There has been much recent interest in ‘impact of research’, what it means, why we need it, and how we measure it. Much of this has focused on academic research, driven by the need to include impact case studies within submissions to the 2014 Research Excellence Framework (REF). However, the impact of education and practice is also of significant importance. In October 2014 the Centre of Postgraduate Medical Research and Education hosted a symposium exploring issues around impact. The event brought together academics, researchers, and clinicians working in healthcare to hear from a range of presenters. A combination of local and national speakers shared their views on impact in their professional and disciplinary areas

Hundley, Vanora

Sheppard, Zoe

Thompson, Paul W.

van Teijlingen, Edwin

Zoë Sheppard, Vanora Hundley, Edwin van Teijlingen and Paul Thompson of Bournemouth University present the challenges of impact in healthcare recently discussed at a symposium held by the Centre of Postgraduate Medical Research and Education at Bournemouth University. Given the imminent results of the Research Excellence Framework 2014, the summarised findings and issues raised on the implementation of impact point to further collaborations needed on the impact agenda in healthcare and beyond

Sheppard, Zoë

Thompson, Paul

LSE Research Online

12/15/2014Collaborative ‘science of science’ needed to ensureresearch and education make a difference to practice.blogs.lse.ac.uk /impactofsocialsciences/2014/12/15/the-impact-agenda-in-healthcare/Zoë Sheppard, Vanora Hundley, Edwin van Teijlingen and Paul Thompson of BournemouthUniversity present the challenges of impact in healthcare recently discussed at asymposium held by the Centre of Postgraduate Medical Research and Education atBournemouth University. Given the imminent results of the Research ExcellenceFramework 2014, the summarised findings and issues raised on the implementation ofimpact point to further collaborations needed on the impact agenda in healthcare andbeyond.There has been much recent interest in ‘impact of research’, what it means, why we need it,and how we measure it. Much of this has focused on academic research, driven by theneed to include impact case studies within submissions to the 2014 Research Excellence Framework (REF).However, the impact of education and practice is also of significant importance. In October 2014 the Centre ofPostgraduate Medical Research and Education hosted a symposium exploring issues around impact. The eventbrought together academics, researchers, and clinicians working in healthcare to hear from a range of presenters. Acombination of local and national speakers shared their views on impact in their professional and disciplinary areas.Reflections on the challenges of impactThe symposium raised many challenges of impact (Table 1) and some of these key challenges are discussed below.Table 1: Summary of challenges around impact and ways going forwardChallenges Way forwardProblematic definitions Awareness raising and cultural changeDifferences in rationale Make impact a strategic priorityDifficulty in determining attribution Audit trail and retrospective appraisal of impactLengthy time lags between research findings andimplementationCollaborative working with clinicians and policymakersTime taken for reporting Open access movementLack of funding Cost impact work into grant applicationsDifficulties in prioritising Involve the public and service usersThe role of marketing Team working with good communicators What do we mean by impact?As all the speakers at the symposium presented their perspectives on impact, it became very clear that the definitionof impact was problematic and that it meant different things to different people – the public, the funder, theUniversity, the provider, the student, and the medical educator. For example, Trish Greenhalgh, one of the keynote1/4speakers, noted that researchers saw impact based around requirements for the REF, as well as the requirementsfor funding organisations. In contrast, the view of the educators of junior doctors, presented by Phil Rushton, wasthat we should identify the impact of teaching and learning on students and organisations and subsequentlypatients. Finally, practitioners saw impact in terms of implementation of National Institute for Health and CareExcellence (NICE) guidelines and infrastructure (e.g. training, facilities, and cost), a view put forward by Paul Lear.Similarly, the rationale for measuring impact was different, ranging from a mechanism for deciding allocation offunds in the REF through to analysis, advocacy, and accountability for the tax paying public, as presented byJonathan Grant, the other keynote speaker. Despite these differences, making sure that research and educationmakes a difference to practice was common to all.Time lagOne of the concerns highlighted during the symposium was the considerable time lag between research beingconducted and findings being implemented into practice, the difference being estimated at 17 years on average .Indeed, the qualitative case study of the bowel cancer test  developed in 1967 presented by Jonathan Grant was notshown to make a health gain until 2010 onwards. Conference participants remarked that this was unacceptable andthat one of the many challenges of the impact agenda is to speed up implementation and increase the value onreturn. Reassuringly, the symposium showcased several excellent examples of research that is making animmediate difference e.g. at a local district general hospital as presented by the Medical Director Paul Lear, andfrom an innovative matched funded studentship to Edd Carlton between the University and a local NHS Trust firmlymaking sure research goes straight into practice.Image credit: Pikiwikisrael, Wikimedia Creative Commons Attribution 2.5 Generic Reporting impactAnother challenge raised during the symposium was the balance between reporting and making an impact. Edwinvan Teijlingen highlighted one retrospective approach taken by Bournemouth University’s School of Health andSocial Care when he highlighted the breastfeeding case study submitted to REF2014. Rather than retrospectivelybuilding case studies, as many universities were forced to do for the REF2014 submission, impact strategies shouldbe built in and costed into grant applications. Indeed, another barrier mentioned was lack of funding as researchgrants are typically under costed in this area in both time and resource. Although systems for reporting are beingimplemented (e.g. ResearchFish), the time taken to report was perceived as a barrier to impact work and noted by2/4Natalie Carter, Head of Research Liaison and Evaluation at Arthritis Research UK. When faced with such trade-offs,an interesting point about prioritisation was made and whether it was best to focus on quantity or quality of impact.Further potential tensions were raised between evaluation and innovation as well as the role of ‘clever marketing’.Lessons learnt moving forwardAlthough the symposium raised many challenges, it also usefully highlighted some of the initiatives that could betaken forward to maximise impact in the future (Table 1). Awareness raising and a cultural shift towards impact willhelp an understanding of what is meant by impact. No doubt the future REF and potential internationalisation maymotivate universities to make impact a strategic priority with infrastructure and partnerships.One of the most important lessons was the need for collaborative working in many different ways. Simon Denegri,Chair of INVOLVE, noted that members of the public and service users need to be even more involved, from‘willingness’ to being ‘active’ in research. Indeed, the Breaking Boundaries review will be emphasising that the futurelies in co-production of knowledge with service users. Furthermore, service users should assist with identifyingoutcomes (e.g. OMERACT and COMET) and setting priorities for research (e.g. James Lind Alliance). It was agreedthat involving service users will mean that research is relevant to them and more likely to make a difference inpractice.The need to involve service users at the reporting stage was also highlighted by Simon Denegri. The challenge ofpoor reporting noted above may be addressed by involving members of the public and service users as reviewers ofresearch as well as on editorial boards of journals. The open access movement is intended to make researchaccessible to people who need it (e.g. patients, students, and researchers in developing countries) meaning it willmore likely be incorporated into education, learning, and practice. Indeed, there is a forthcoming patient and publicinvolvement journal soon to be launched which will be open access.Edwin van Teijlingen also reminded the audience that for the REF not every piece of research has to have ameasurable impact as one impact case study was required for every ten academics submitted to the REF. This is toensure that universities do undertake speculative, theoretical, or ‘blue sky’ research that might not lead to anyimmediate useful applications.The team approach is also useful in the communicating and sharing of information so that those with goodcommunication skills can be utilised to get findings out. It was clear that academics and clinicians in practice needeach other in order to make research translational and so they should not remain in their isolated silos. In order toimplement research into practice, decision makers and policy makers also need to be on research teams. Indeed ateam approach may influence the disciplines and hence the target Units of Assessment submitted to in the next REF.ConclusionAs noted by Jonathan Grant, one of the keynote speakers, the symposium made it clear that a ‘science of science’ isneeded to understand what works in terms of impact (e.g. implementation science). Despite the many challenges,the impact agenda brings yet more argument for a collaborative team approach. Another of the keynote speakers,Trish Greenhalgh, presented a call for action around supplementing ‘high church’ academic research with clinicallyled research, contract research, curiosity driven research that is co-produced. The symposium importantly raisedawareness of these issues and is hopefully a step forward towards raising the impact agenda for academics,educationalists, and clinicians working in the healthcare field. We welcome yet more debate around these issues tofurther again promote awareness.Acknowledgements: The authors would like to thank the speakers and panel facilitators at the symposium (TimBattcock, Keith Brown, Edward Carlton, Natalie Carter, Simon Denegri, John Fletcher, Jonathan Grant, TrishGreenhalgh, Paul Lear, and Phil Rushton) for sharing their views, as well as the participants, which have formed thebasis of this discussion piece. Thanks also to Lisa Gale for reading an early draft.3/4Note: This article gives the views of the authors, and not the position of the Impact of Social Science blog, nor of theLondon School of Economics. Please review our Comments Policy if you have any concerns on posting a commentbelow.About the AuthorsZoë Sheppard is a Research Fellow in Research Methods at Bournemouth UniversityVanora Hundley is Deputy Dean for Research and Professional Practice at Bournemouth UniversityEdwin van Teijlingen is Professor of Maternal and Perinatal Health Research at Bournemouth UniversityPaul Thompson is Director of the Centre of Postgraduate Medical Research and Education at BournemouthUniversity.Copyright © The Author (or The Authors) - Unless otherwise stated, this work is licensed under a CreativeCommons Attribution Unported 3.0 License.4/4

Collaborative ‘science of science’ needed to ensure research and education make a difference to practice.

Bournemouth University Research Online

blogs.lse.ac.uk http://blogs.lse.ac.uk/impactofsocialsciences/2014/12/15/the-impact-agenda-in-healthcare/Collaborative ‘science of science’ needed to ensure researchand education make a difference to practice.Zoë Sheppard, Vanora Hundley, Edwin van Teijlingen and Paul Thompson ofBournemouth University present the challenges of impact in healthcare recentlydiscussed at a symposium held by the Centre of Postgraduate Medical Research andEducation at Bournemouth University. Given the imminent results of the ResearchExcellence Framework 2014, the summarised findings and issues raised on theimplementation of impact point to further collaborations needed on the impact agenda inhealthcare and beyond.There has been much recent interest in ‘impact of research’, what it means, why we needit, and how we measure it. Much of this has focused on academic research, driven by the need to include impactcase studies within submissions to the 2014 Research Excellence Framework (REF). However, the impact ofeducation and practice is also of significant importance. In October 2014 the Centre of Postgraduate MedicalResearch and Education hosted a symposium exploring issues around impact. The event brought togetheracademics, researchers, and clinicians working in healthcare to hear from a range of presenters. A combination oflocal and national speakers shared their views on impact in their professional and disciplinary areas.Reflections on the challenges of impactThe symposium raised many challenges of impact (Table 1) and some of these key challenges are discussedbelow.Table 1: Summary of challenges around impact and ways going forwardChallenges Way forwardProblematic definitions Awareness raising and cultural changeDifferences in rationale Make impact a strategic priorityDifficulty in determining attribution Audit trail and retrospective appraisal of impactLengthy time lags between research findings andimplementationCollaborative working with clinicians and policymakersTime taken for reporting Open access movementLack of funding Cost impact work into grant applicationsDifficulties in prioritising Involve the public and service usersThe role of marketing Team working with good communicators What do we mean by impact?As all the speakers at the symposium presented their perspectives on impact, it became very clear that thedefinition of impact was problematic and that it meant different things to different people – the public, the funder,the University, the provider, the student, and the medical educator. For example, Trish Greenhalgh, one of thekeynote speakers, noted that researchers saw impact based around requirements for the REF, as well as therequirements for funding organisations. In contrast, the view of the educators of junior doctors, presented by PhilRushton, was that we should identify the impact of teaching and learning on students and organisations andsubsequently patients. Finally, practitioners saw impact in terms of implementation of National Institute for Healthand Care Excellence (NICE) guidelines and infrastructure (e.g. training, facilities, and cost), a view put forward byPaul Lear. Similarly, the rationale for measuring impact was different, ranging from a mechanism for decidingallocation of funds in the REF through to analysis, advocacy, and accountability for the tax paying public, aspresented by Jonathan Grant, the other keynote speaker. Despite these differences, making sure that researchand education makes a difference to practice was common to all.Time lagOne of the concerns highlighted during the symposium was the considerable time lag between research beingconducted and findings being implemented into practice, the difference being estimated at 17 years on average .Indeed, the qualitative case study of the bowel cancer test  developed in 1967 presented by Jonathan Grant wasnot shown to make a health gain until 2010 onwards. Conference participants remarked that this wasunacceptable and that one of the many challenges of the impact agenda is to speed up implementation andincrease the value on return. Reassuringly, the symposium showcased several excellent examples of researchthat is making an immediate difference e.g. at a local district general hospital as presented by the Medical DirectorPaul Lear, and from an innovative matched funded studentship to Edd Carlton between the University and a localNHS Trust firmly making sure research goes straight into practice.Image credit: Pikiwikisrael, Wikimedia Creative Commons Attribution 2.5 Generic Reporting impactAnother challenge raised during the symposium was the balance between reporting and making an impact. Edwinvan Teijlingen highlighted one retrospective approach taken by Bournemouth University’s School of Health andSocial Care when he highlighted the breastfeeding case study submitted to REF2014. Rather than retrospectivelybuilding case studies, as many universities were forced to do for the REF2014 submission, impact strategiesshould be built in and costed into grant applications. Indeed, another barrier mentioned was lack of funding asresearch grants are typically under costed in this area in both time and resource. Although systems for reportingare being implemented (e.g. ResearchFish), the time taken to report was perceived as a barrier to impact workand noted by Natalie Carter, Head of Research Liaison and Evaluation at Arthritis Research UK. When faced withsuch trade-offs, an interesting point about prioritisation was made and whether it was best to focus on quantity orquality of impact. Further potential tensions were raised between evaluation and innovation as well as the role of‘clever marketing’.Lessons learnt moving forwardAlthough the symposium raised many challenges, it also usefully highlighted some of the initiatives that could betaken forward to maximise impact in the future (Table 1). Awareness raising and a cultural shift towards impact willhelp an understanding of what is meant by impact. No doubt the future REF and potential internationalisation maymotivate universities to make impact a strategic priority with infrastructure and partnerships.One of the most important lessons was the need for collaborative working in many different ways. Simon Denegri,Chair of INVOLVE, noted that members of the public and service users need to be even more involved, from‘willingness’ to being ‘active’ in research. Indeed, the Breaking Boundaries review will be emphasising that thefuture lies in co-production of knowledge with service users. Furthermore, service users should assist withidentifying outcomes (e.g. OMERACT and COMET) and setting priorities for research (e.g. James Lind Alliance).It was agreed that involving service users will mean that research is relevant to them and more likely to make adifference in practice.The need to involve service users at the reporting stage was also highlighted by Simon Denegri. The challenge ofpoor reporting noted above may be addressed by involving members of the public and service users as reviewersof research as well as on editorial boards of journals. The open access movement is intended to make researchaccessible to people who need it (e.g. patients, students, and researchers in developing countries) meaning it willmore likely be incorporated into education, learning, and practice. Indeed, there is a forthcoming patient andpublic involvement journal soon to be launched which will be open access.Edwin van Teijlingen also reminded the audience that for the REF not every piece of research has to have ameasurable impact as one impact case study was required for every ten academics submitted to the REF. This isto ensure that universities do undertake speculative, theoretical, or ‘blue sky’ research that might not lead to anyimmediate useful applications.The team approach is also useful in the communicating and sharing of information so that those with goodcommunication skills can be utilised to get findings out. It was clear that academics and clinicians in practice needeach other in order to make research translational and so they should not remain in their isolated silos. In order toimplement research into practice, decision makers and policy makers also need to be on research teams. Indeeda team approach may influence the disciplines and hence the target Units of Assessment submitted to in the nextREF.ConclusionAs noted by Jonathan Grant, one of the keynote speakers, the symposium made it clear that a ‘science of science’is needed to understand what works in terms of impact (e.g. implementation science). Despite the manychallenges, the impact agenda brings yet more argument for a collaborative team approach. Another of thekeynote speakers, Trish Greenhalgh, presented a call for action around supplementing ‘high church’ academicresearch with clinically led research, contract research, curiosity driven research that is co-produced. Thesymposium importantly raised awareness of these issues and is hopefully a step forward towards raising theimpact agenda for academics, educationalists, and clinicians working in the healthcare field. We welcome yetmore debate around these issues to further again promote awareness.Acknowledgements: The authors would like to thank the speakers and panel facilitators at the symposium (TimBattcock, Keith Brown, Edward Carlton, Natalie Carter, Simon Denegri, John Fletcher, Jonathan Grant, TrishGreenhalgh, Paul Lear, and Phil Rushton) for sharing their views, as well as the participants, which have formedthe basis of this discussion piece. Thanks also to Lisa Gale for reading an early draft.Note: This article gives the views of the authors, and not the position of the Impact of Social Science blog, nor ofthe London School of Economics. Please review our Comments Policy if you have any concerns on posting acomment below.About the AuthorsZoë Sheppard is a Research Fellow in Research Methods at Bournemouth UniversityVanora Hundley is Deputy Dean for Research and Professional Practice at Bournemouth UniversityEdwin van Teijlingen is Professor of Maternal and Perinatal Health Research at Bournemouth UniversityPaul Thompson is Director of the Centre of Postgraduate Medical Research and Education at BournemouthUniversity.Copyright 2011 LSE Impact of Social Sciences

Collaborative ‘science of science’ needed to ensure research and education make a difference to practice.

Abstract

Similar works

Full text

Available Versions

LSE Research Online

Bournemouth University Research Online