A dearth of clear, relevant and reliable research evidence continue to block the use of research, according to a study of 145 research papers on evidence use. According to the authors of the review, Kathryn Oliver, Simon Innvær, Theo Lorenc, Jenny Woodman, and James Thomas difficulty finding and accessing this research is also a major problem

Innvaer, Simon

Lorenc, Theo

Oliver, Kathryn

Thomas, James

Oliver, K

Innvaer, S

Lorenc, T

Thomas, J

LSHTM Research Online

Negative stereotypes about the policymaking process hinderproductive action toward evidence-based policy.A dearth of clear, relevant and reliable research evidence continue to block the useof research, according to a study of 145 research papers on evidence use.According to the authors of the review, Kathryn Oliver, Simon Innvær, TheoLorenc, Jenny Woodman, and James Thomas difficulty finding and accessing thisresearch is also a major problem. Despite several decades of work on evidence-based policy, the goals of improving researchuptake and promoting greater use of research within policy making are still elusive.  Academicinterest in the area grew out of Evidence-Based Medicine, dating back to Archie Cochrane’s bookEffectiveness and Efficiency: Random Reflections on Health Services published in 1972. Todescribe the current body of literature in the area, we carried out a systematic review, updating anearlier study of the reasons why policy-makers use (or don’t use) evidence. We found 145 papersfrom countries all around the world, covering a wide range of policy areas, from health to criminaljustice, to transport, to environmental conservation. We have concluded our findings below in thehopes of reducing the barriers of evidence-based policy.Source: A systematic review of barriers to and facilitators of the use of evidence by policymakers (2014)What are the major barriers to policymakers using evidence?Despite these very different contexts, the barriers and facilitators of research were remarkablysimilar. Researchers, for example, found that poor access to research, dissemination and costswere still a problem. A dearth of clear, relevant and reliable research evidence, and difficultyfinding and accessing it, were the main barriers to the use of research. Access to research,managerial and organisational resources to support finding and using research were all importantfactors.Institutions and formal organisations themselves have an important role to play. For example,decision-making bodies often have formal processes by which to consult on and take decisions.Participation of organisations such as NICE can help research to be part of a decision, but manystudy participants saw the influence of vested interest groups and lobbyists as an obstacle to theuse of evidence. The production of guidelines by professional organisations was also noted in thereview as a possible facilitator of evidence use by policy makers; however, in cases where theorganisation was itself regarded as not being influential (such as the WHO) this was not the case.Whether true or not, these perceptions can damage an organisation’s ability to change practice orpolicy. On the other hand, institutions can make a positive difference by providing clear leadershipto champion evidence-based policy.Evidence comes from people we know, not journalsUnsurprisingly, good relationships and collaborations between researchers and policymakers helpresearch use. This is a common finding—we already knew that policymakers often prefer to getinformation and advice from friends and colleagues, rather than papers and journals. Oneconsequence is that policymakers may take advice from academics they already know throughuniversity or school. In the UK, that tends to mean Oxbridge, LSE, UCL and Kings.In our everyday lives, we seek advice from friends and colleagues; we don’t trust people who havebeen wrong before, or who seem to be biased – or with whom we just don’t get along. Exactly thesame applies to the process of policymaking and, in our experience, in the process of researchcollaboration. Negative stereotypes abound on both sides, and our review found that personalexperiences, judgments, and values were important factors in whether evidence was used.Policymakers and researchers of course have very different pressures, working environments,demands, needs, career paths and merit structures. Their definitions of ‘evidence’, ‘use’ and‘reliability’ are also likely to differ. Hence, they have very different perceptions of the value ofresearch for policy-making.However, while these differing perceptions are well documented, we know surprisingly little aboutwhat policy-makers actually do with research. This gap helps to perpetuate academics’ commonlyheld view that policymakers do not use evidence—a negative stereotype that, as any politician willtell you, is hardly conducive to trust or collaboration. As a result, we still do not understand how toimprove research impact on policy – or, indeed, why we should try.Do definitions matter?Policymakers tend to have a broader definition of evidence than that usually accepted byacademics. Academic researchers, understandably, tend to think of ‘evidence’ as academicresearch findings, while policy-makers often use and value other types of evidence. For example,over a third of the studies mentioned that use of informal evidence such as local data or tacitknowledge.Other factors reported to facilitate use of evidence are timely access to good quality and relevantresearch evidence. However, ‘making policy’ and ‘using evidence’ are both really difficult to do andto understand. These are not simple, linear processes Linear Stages of Policy Process from the Rational Framework. Adapted from Policy Analysis: A Political and OrganizationalPerspective by William Jenkins (1978, p. 17).Figure 1: Examples of models of the policy process found in academia. Source: Wellcome Trust Educational Resources.But nor are they repeatable, predictable cycles of events. The reality is likely to be far morecomplicated. The role of chance, e.g. who was available that day, or who you happened to have aconversation with, is hard to underestimate.It is worth asking why most policymakers are not prioritising overcoming these barriersthemselves. It is possible – even likely – that politicians do not want their pet policies underminedby evidence. However, we believe the research on this question—as well as much of the debateamong policy-makers and commentators about research impact and evidence-based policy—isbased on the three flawed assumptions that: 1) Policymakers do not use evidence 2)Policymakers should use more evidence and 3) If research evidence had more impact, policywould be better.In fact, these assumptions tell us more about academics than about policy. Policy uses evidenceall the time—just not necessarily research evidence.  Academics used to giving 45-minuteseminars do not always understand that a hard-pressed policymaker would prefer a 20-secondphone call. Nor do researchers understand the types of legitimate evidence, such as publicopinion, political feasibility and knowledge of local contexts, that are essential to understandinghow policies happen.There are a great many finger-wagging papers telling policymakers to ‘upskill’, but few are urgingacademics and researchers to learn about the policy process. What are the incentives forpolicymakers to try and engage with science and evidence, when it takes too long, is often poorquality, and they are told they are not skilled enough to understand it?There are important unanswered questions relating to the use of research in policy. We don’t knowhow evidence actually impacts policy, or how policy-makers ‘use’ evidence—to create options?Defeat opponents? Strengthen bargaining positions? What is the role of interpersonal interactionin policy? Finally, does increasing research impact actually make for better policy, whatever thismeans? Academics have any number of stories about poor policies that ignore research, but lessin the way of rigorous evidence showing the beneficial impact of research.Other countries have tried creating formal organisations and spaces for relationships to flourish.Inthe USA, for an academic to leave a University and work at an organisation such as Brookings isseen as a promotion, not a failure. Australia’s Sax Institute provides a forum for policymakers andacademics come together, similar to the ideas behind Cambridge’s Centre for Science and Policy.It remains to be seen how influential these organisations will be, and whether they succeed inchanging the stereotypes and behaviours of both policymakers and academics. And critically,whether relationships brokered in this way can transcend the traditional ‘expert advice’ model ofresearch mobilisation to encompass evidence that aims to be an accurate view of the totality ofcurrent knowledge on a problem – like this systematic review.In the meantime, rigorous studies ofthe three common misconceptions about evidence-based policy will help us all understand howand whether we should get which evidence into policy. Solutions are better than excuses.This blog has been previously published by Research Fortnight and the Alliance for UsefulEvidence.Featured Image credit: U.S. Department of State (Wikimedia, public domain)Note: This article gives the views of the authors, and not the position of the Impact of SocialScience blog, nor of the London School of Economics. Please review our Comments Policy if youhave any concerns on posting a comment below.About the AuthorsKathryn Oliver is a Provost Fellow in the Department of Science, Technology, Engineering andPublic Policy (STEaPP) at UCL. She is interested in the debates about evidence-based policy, therole of policy networks, and ways of thinking about the impact of research on policy.Simon Innvær is an associate professor at Oslo University College, Norway. He is interested inevidence based policy, and the combination of process and effect evaluations.Theo Lorenc is Provost Fellow in the Department of Science, Technology, Engineering and PublicPolicy (STEaPP) at UCL. His research focuses on interactions between knowledge communities.James Thomas is Assistant Director for Health & Wellbeing at the Institute of Education inLondon and Director of the EPPI-Centre’s Reviews Facility for the Department of Health, England,which undertakes systematic reviews across a range of policy areas to support the department.He specialises in developing methods for research synthesis, in particular for qualitative andmixed methods reviews and in using emerging information technologies such as text mining inresearch. Prof Thomas leads a module on synthesis and critical appraisal on the EPPI-Centre’sMSc in Research for Public Policy and Practice and development on the Centre’s in-housereviewing software, EPPI-Reviewer.June 2nd, 2014 | Evidence-based policy, Government, Impact, Knowledge transfer, Publicengagement | 15 CommentsShare This Story, Choose Your Platform!This work by LSE Impact of Social Sciences blog is licensed under a Creative Commons Attribution 3.0Unported.

Negative stereotypes about the policymaking process hinder productive action toward evidence-based policy.

LSE Research Online

6/2/2014Negative stereotypes about the policymaking processhinder productive action toward evidence-based policy.blogs.lse.ac.uk /impactofsocialsciences/2014/06/02/how-to-get-policymakers-to-use-more-evidence/A dearth of clear, relevant and reliable research evidence continue to block the use of research, according toa study of 145 research papers on evidence use. According to the authors of the review, KathrynOliver, Simon Innvær, Theo Lorenc, Jenny Woodman, and James Thomas difficulty finding andaccessing this research is also a major problem. Despite several decades of work on evidence-based policy, the goals of improving research uptakeand promoting greater use of research within policy making are still elusive.  Academic interest inthe area grew out of Evidence-Based Medicine, dating back to Archie Cochrane’s bookEffectiveness and Efficiency: Random Reflections on Health Services published in 1972. Todescribe the current body of literature in the area, we carried out a systematic review, updating an earlier study ofthe reasons why policy-makers use (or don’t use) evidence. We found 145 papers from countries all around theworld, covering a wide range of policy areas, from health to criminal justice, to transport, to environmentalconservation. We have concluded our findings below in the hopes of reducing the barriers of evidence-based policy.Source: A systematic review of barriers to and facilitators of the use of evidence bypolicymakers (2014)1/5What are the major barriers to policymakers using evidence?Despite these very different contexts, the barriers and facilitators of research were remarkably similar. Researchers,for example, found that poor access to research, dissemination and costs were still a problem. A dearth of clear,relevant and reliable research evidence, and difficulty finding and accessing it, were the main barriers to the use ofresearch. Access to research, managerial and organisational resources to support finding and using research wereall important factors.Institutions and formal organisations themselves have an important role to play. For example, decision-makingbodies often have formal processes by which to consult on and take decisions. Participation of organisations suchas NICE can help research to be part of a decision, but many study participants saw the influence of vested interestgroups and lobbyists as an obstacle to the use of evidence. The production of guidelines by professionalorganisations was also noted in the review as a possible facilitator of evidence use by policy makers; however, incases where the organisation was itself regarded as not being influential (such as the WHO) this was not the case.Whether true or not, these perceptions can damage an organisation’s ability to change practice or policy. On theother hand, institutions can make a positive difference by providing clear leadership to champion evidence-basedpolicy.Evidence comes from people we know, not journalsUnsurprisingly, good relationships and collaborations between researchers and policymakers help research use.This is a common finding—we already knew that policymakers often prefer to get information and advice fromfriends and colleagues, rather than papers and journals. One consequence is that policymakers may take advicefrom academics they already know through university or school. In the UK, that tends to mean Oxbridge, LSE, UCLand Kings.In our everyday lives, we seek advice from friends and colleagues; we don’t trust people who have been wrongbefore, or who seem to be biased – or with whom we just don’t get along. Exactly the same applies to the process ofpolicymaking and, in our experience, in the process of research collaboration. Negative stereotypes abound on bothsides, and our review found that personal experiences, judgments, and values were important factors in whetherevidence was used.Policymakers and researchers of course have very different pressures, working environments, demands, needs,career paths and merit structures. Their definitions of ‘evidence’, ‘use’ and ‘reliability’ are also likely to differ. Hence,they have very different perceptions of the value of research for policy-making.However, while these differing perceptions are well documented, we know surprisingly little about what policy-makers actually do with research. This gap helps to perpetuate academics’ commonly held view that policymakersdo not use evidence—a negative stereotype that, as any politician will tell you, is hardly conducive to trust orcollaboration. As a result, we still do not understand how to improve research impact on policy – or, indeed, why weshould try.Do definitions matter?Policymakers tend to have a broader definition of evidence than that usually accepted by academics. Academicresearchers, understandably, tend to think of ‘evidence’ as academic research findings, while policy-makers oftenuse and value other types of evidence. For example, over a third of the studies mentioned that use of informalevidence such as local data or tacit knowledge.Other factors reported to facilitate use of evidence are timely access to good quality and relevant research evidence.However, ‘making policy’ and ‘using evidence’ are both really difficult to do and to understand. These are not simple,linear processes2/5 LinearStagesofPolicyProcess from the Rational Framework. Adapted from Policy Analysis: A Political andOrganizational Perspective by William Jenkins (1978, p. 17).Figure 1: Examples of models of the policy process found in academia. Source: WellcomeTrust Educational Resources.But nor are they repeatable, predictable cycles of events. The reality is likely to be far more complicated. The role ofchance, e.g. who was available that day, or who you happened to have a conversation with, is hard tounderestimate.It is worth asking why most policymakers are not prioritising overcoming these barriers themselves. It is possible –even likely – that politicians do not want their pet policies undermined by evidence. However, we believe theresearch on this question—as well as much of the debate among policy-makers and commentators about researchimpact and evidence-based policy—is based on the three flawed assumptions that: 1) Policymakers do not useevidence 2) Policymakers should use more evidence and 3) If research evidence had more impact, policy would bebetter.In fact, these assumptions tell us more about academics than about policy. Policy uses evidence all the time—justnot necessarily research evidence.  Academics used to giving 45-minute seminars do not always understand that ahard-pressed policymaker would prefer a 20-second phone call. Nor do researchers understand the types of3/5legitimate evidence, such as public opinion, political feasibility and knowledge of local contexts, that are essential tounderstanding how policies happen.There are a great many finger-wagging papers telling policymakers to ‘upskill’, but few are urging academics andresearchers to learn about the policy process. What are the incentives for policymakers to try and engage withscience and evidence, when it takes too long, is often poor quality, and they are told they are not skilled enough tounderstand it?There are important unanswered questions relating to the use of research in policy. We don’t know how evidenceactually impacts policy, or how policy-makers ‘use’ evidence—to create options? Defeat opponents? Strengthenbargaining positions? What is the role of interpersonal interaction in policy? Finally, does increasing research impactactually make for better policy, whatever this means? Academics have any number of stories about poor policiesthat ignore research, but less in the way of rigorous evidence showing the beneficial impact of research.Other countries have tried creating formal organisations and spaces for relationships to flourish.In the USA, for anacademic to leave a University and work at an organisation such as Brookings is seen as a promotion, not a failure.Australia’s Sax Institute provides a forum for policymakers and academics come together, similar to the ideasbehind Cambridge’s Centre for Science and Policy.It remains to be seen how influential these organisations will be, and whether they succeed in changing thestereotypes and behaviours of both policymakers and academics. And critically, whether relationships brokered inthis way can transcend the traditional ‘expert advice’ model of research mobilisation to encompass evidence thataims to be an accurate view of the totality of current knowledge on a problem – like this systematic review.In themeantime, rigorous studies of the three common misconceptions about evidence-based policy will help us allunderstand how and whether we should get which evidence into policy. Solutions are better than excuses.This blog has been previously published by Research Fortnight and the Alliance for Useful Evidence.Featured Image credit: U.S. Department of State (Wikimedia, public domain)Note: This article gives the views of the authors, and not the position of the Impact of Social Science blog, nor of theLondon School of Economics. Please review our Comments Policy if you have any concerns on posting a commentbelow.About the AuthorsKathryn Oliver is a Provost Fellow in the Department of Science, Technology, Engineering and Public Policy(STEaPP) at UCL. She is interested in the debates about evidence-based policy, the role of policy networks, andways of thinking about the impact of research on policy.Simon Innvær is an associate professor at Oslo University College, Norway. He is interested in evidence basedpolicy, and the combination of process and effect evaluations.Theo Lorenc is Provost Fellow in the Department of Science, Technology, Engineering and Public Policy (STEaPP)at UCL. His research focuses on interactions between knowledge communities.James Thomas is Assistant Director for Health & Wellbeing at the Institute of Education in London and Director ofthe EPPI-Centre’s Reviews Facility for the Department of Health, England, which undertakes systematic reviewsacross a range of policy areas to support the department. He specialises in developing methods for researchsynthesis, in particular for qualitative and mixed methods reviews and in using emerging information technologiessuch as text mining in research. Prof Thomas leads a module on synthesis and critical appraisal on the EPPI-Centre’s MSc in Research for Public Policy and Practice and development on the Centre’s in-house reviewingsoftware, EPPI-Reviewer.4/5Copyright © The Author (or The Authors) - Unless otherwise stated, this work is licensed under a CreativeCommons Attribution Unported 3.0 License.5/5

http://researchonline.lshtm.ac.uk/4646374/1/Negative%20stereotypes%20about%20the%20policymaking%20process%20hinder%20productive%20action%20toward%20evidence-based%20policy.pdf

Negative stereotypes about the policymaking process hinder productive action toward evidence-based policy.

Abstract

Similar works

Full text

Available Versions

LSHTM Research Online

LSE Research Online