The quality of scientific evidence in government heavily depends upon the independent assessment of research. Pressure from those commissioning the research may pose a threat to scientific integrity and rigorous policy-making. Edward Page reports that whilst there is strong evidence of government leaning, this leaning appears to have little systematic impact on the nature of the conclusions that researchers reach due to the presence of disincentives within academia and research administrators within government

Page, Edward

LSE Research Online

Image credit: Andy Arthur (CC-BY)10/31/2013There is sufficient evidence to suggest Whitehall is leaningon researchers to produce politically useful research.blogs.lse.ac.uk /impactofsocialsciences/2013/10/31/governments-lean-on-researchers/The quality of scientific evidence in government heavily depends upon the independent assessment ofresearch. Pressure from those commissioning the research may pose a threat to scientific integrityand rigorous policy-making. Edward Page reports that whilst there is strong evidence ofgovernment leaning, this leaning appears to have little systematic impact on the nature of theconclusions that researchers reach due to the presence of disincentives within academia andresearch administrators within government.Do governments lean too much on the researchers who evaluate their policies? One can think ofone reason why they would: to get good publicity. And one can think of a reason why researcherswould give in to such pressure: to get contracts. But there are also plausible reasons to expect government not tolean on researchers: whether government genuinely wants to draw lessons from research or just wants good PR itneeds to be rigorous rather than obsequious. Moreover researchers might not be expected to give in to pressuresince their reputations are built on independence and can be destroyed by evidence or suspicions that theirprofessional opinions are for sale.The issue is an important one even though the signs are that theCoalition government is not as keen as New Labour was on securingevaluation evidence by commissioning research. If evaluation researchis generated under pressure from sponsors intending to produce resultsthat suit them, then the nature of this pressure and the responses ofthose facing it are at the very least relevant to our assessment of thequality of that evidence. To examine government leaning and researcherbuckling we conducted a study, based on a web survey of 204academics who had done research work for government since 2005,supplemented by interviews with 22 researchers.The strongest evidence of government influence can be found in thevery initial stages of the research: setting up the research design. Someof our respondents offered comments along these lines:… the real place where research is politically managed is in the selection of topics/areas to beresearched and then in the detailed specification. It is there that they control the kinds of questionsthat are to be asked. This gives plenty of opportunity to avoid difficult results.When asked, nearly half of our respondents (45 per cent) pointed out that “the government organisation(s) had aclear idea of the precise questions the research should examine” with 27 per cent suggesting “the governmentorganisation(s) had a broad idea of what the research should examine and left the definition of the precise researchquestions to me/us”, 26 per cent indicated the development of the research questions was a joint effort withgovernment and 2 per cent did not know.1/3Source:  “Evaluation Under Contract: Government Pressure and the Production of Policy Research” Pre-print [PDF]Whether government intervened in the research design or not appears to have a significant influence on whether theresearch produced was supportive of government policy or not, as reported by our respondents. Of those who wereleft to develop the research questions, only 23 per cent produced a report broadly supportive of government policycompared with 50 per cent of those writing reports.  When government was involved in developing researchquestions, whether alone or in conjunction with researchers, their reports were substantially more likely to besupportive.At no other stage in the research did government pressure have an impact on how supportive the final report turnedout to be.  This was not for want of trying. For instance,   52 per cent reported they were asked to make significantchanges to their draft reports (i.e. affecting the interpretation of findings or the weight given to them). And some ofour respondents elaborated along these lines: “There was a lot of dialogue back and forth at the end between usand the Department … before it was published to ensure they did not look bad. They wanted certain wordingchanged so that it was most beneficial from a PR and marketing point of view; and they wanted certain thingsemphasised and certain things omitted”. Yet such pressure seems to have had little effect on whether the end resultwas a favourable or critical report. If anything, those asked to make changes were more likely to produce criticalreports (though this finding does not approach statistical significance).One must be careful about what we mean by “government” influence. There are at least four distinct groups within aministry, each with different relationships with researchers. First there are the officials who are responsible forresearch, possibly because they are researchers themselves.  Where mentioned, these seem to have the bestrelationships with researchers and appear most likely to share a belief in the importance of programme evaluationobjectives in research. A second group is made up of “policy people” — officials with the task of looking after policywithin the department, whether to amend, defend, expand or contract it.  One respondent summarised concisely herview of the difference between these two groups: “the research manager places a lot of emphasis on researchintegrity, whereas the policy teams may have their own ideological or policy motives”.The third group is the ministerial political leadership. Our survey evidence suggests unsurprisingly that they arehighly likely, in the view of our respondents, to downvalue programme evaluation objectives of research as only 4per cent of respondents (N=182) agreed with the proposition that “Ministers are prepared to act on evidenceprovided by social research even when it runs counter to their views”.  The professionals and service providers inthe programmes being evaluated make up a fourth group.  Their views might be sought on an ad hoc basis as theresearch develops or they might be part of “stakeholder” or “expert” steering groups. Several respondents andinterviewees mentioned the role of service providers as a source of constraint on their research findings.  Oneargued2/3We met regularly with the Head of Research in the [Department] and also occasionally with theirpolicymaking colleagues.  One difficulty with these meetings was that they insisted representatives ofthe [organisation running programme being evaluated] attended. This made it quite difficult to discussthe report openly because these peoples’ livelihoods depended on the scheme. My part of the reportwas critical of the [programme] and I thought it inappropriate for the [department] to invite thesepeople along.  I felt it hindered honest and open discussion.Overall there is sufficient evidence here to suggest that governments do lean on researchers. However, for the mostpart this leaning appears to have little systematic impact on the nature of the conclusions that researchers reach.The most effective constraint appears to be found when government specifies the nature of the research to be doneat the outset. No other form of constraint has as powerful an effect on the degree to which the overall conclusionsthe researchers reach support government policy.Our findings suggest that there are two main forces that reduce the impact of government “leaning” on the characterand quality of the research reports.  The first is the persistence of disincentives within academic career structures tocompromise scientific integrity for the sake of securing government contracts.  Our findings point to this, but we alsohave to note the shortcomings of our evidence base in this respect: it is based on academics reporting on their ownbehaviour. The second is the existence within government of a body of research administrators given significantresponsibility for developing and managing research and coming in between policy officials and politicians on theone hand and the researchers on the other. In the absence of such a body of research administrators, the pressureson researchers to produce politically congenial research would likely be far stronger.  Without the disincentives tocompromise scientific integrity, one would have to have serious concern for the value of commissioned research.This research by the LSE GV314 Group was recently published in the journal Public Administration . GV314Empirical Research in Government is a final year undergraduate course in the Government Department atLSE.  With a group of up to 15 students, Edward Page conducts a separate research project each year. More detailson the project can be found here.Note: This article gives the views of the author, and not the position of the Impact of Social Science blog, nor of theLondon School of Economics. Please review our Comments Policy if you have any concerns on posting a commentbelow.About the AuthorProfessor Edward Page FBA is the Sidney and Beatrice Webb Professor of Public Policy in the Department ofGovernment at LSE. His most recent books are Changing government relations in Europe (London: Routledge,2010), co-edited with Mike Goldsmith; and Policy Bureaucracy. Government with a cast of thousands (Oxford:Oxford University Press, 2005). Co-authored with B. Jenkins.Copyright © The Author (or The Authors) - Unless otherwise stated, this work is licensed under a CreativeCommons Attribution Unported 3.0 License.3/3

There is sufficient evidence to suggest Whitehall is leaning on researchers to produce politically useful research.

Abstract

Similar works

Full text

Available Versions

LSE Research Online