IAISE Policy Brief No. 1 on "Impact Assessment Systems and Tools in Europe:
Current Practices and User Expectations" has just been published. This policy
brief presents the key findings on the most comprehensive survey yet conducted
of user needs and expectations with regard to Impact Assessment (IA) systems
and tools in 17 European countries. The survey was carried out by researchers
from the LIAISE Network of Excellence, who collected data through documentary
analysis and interviews with 130 people who steer IA at a strategic level,
i.e. those people who champion, oversee, guide, audit or write guidance for IA
processes. A more detailed description of the survey’s results can be found in
the LIAISE Innovation Report No. 2 (January 2011)

Adelle, Camilla

Moulogianni, Christina

Soba, Marta Perez

Sánchez, Begoña

Institutional Repository of the Freie Universität Berlin

       LIAISE Policy Brief This policy brief presents the key findings on the most comprehensive survey yet conducted of user needs and expectations with regard to Impact Assessment (IA) systems and tools in 17 European countries (namely, Belgium; the Czech Republic; Cyprus; Denmark; Estonia; Finland; Germany; Greece; Ireland; Italy; Lithuania; the Netherlands; Poland; Spain; Sweden; Switzerland; and the UK).   The survey was carried out by researchers from the LIAISE Network of Excellence, who collected data through documentary analysis and interviews with 130 people who steer IA at a strategic level, i.e. those people who champion, oversee, guide, audit or write guidance for IA processes.  A more detailed description of the survey’s results can be found in the LIAISE INNOVATION REPORT N° 2 dated January 2011.  Authors: Camilla Adelle, Christina Moulogianni, Marta Perez Soba, Begoña Sánchez.  n.1 October 2012Systems and tools in Europe: Current practices and user expectations    Policy level appraisal or Impact Assessment (IA) seeks to inform decision makers by predicting and evaluating the potential impacts of policy options. Over the last decade, IA has experienced an enormous global expansion, and is now practiced in all OECD member countries. Within the European Union, it arrived on the political agenda of the Member States and the European Commission in the late 1990s.   Nowadays, IA is regarded as the cornerstone of programmes for better regulation, acting as a key mechanism to improve the quality of regulation as well as to integrate different policy objectives.   However, the widespread diffusion of IA has not necessarily produced a convergence in IA practices: there is a wide variety of IA systems, with different institutional set-ups, objectives and operating cultures. Consequently, understanding the practice and principles of IA requires a full consideration of its diversity. This is equally true when considering the use of IA tools (such as Cost Benefit Analysis, or computer-based modelling) in the IA process, which is one of the key foci of the LIAISE network.     Key findings  There are multiple approaches to Impact Assessment The Impact Assessment (IA) systems (i.e. the mechanisms, structures and processes for implementing IA), the purposes for employing them, and the tools they use vary both within and between countries. Many different factors affect the way they are structured and their functioning. These include the availability of resources (skills, time and data with which to conduct an IA) as well as the quality control mechanisms put in place.   Although many countries have sought to learn from one another, and from international bodies such as the OECD, there is still no one dominant approach to undertaking IA. Rather, each country employs IA in a distinctive way which fits its prevailing political and policy context. It is important therefore not to ‘de-contextualise’ IA, especially when seeking to define and extend ‘best practices’ or increase the use of IA tools.   Impact Assessment tool use is not (yet) institutionalised Many - but by no means all – IA systems already harness the analytical power of IA tools to inform their assessment activities. 10 if the 17 countries surveyed actively promote tool use via the production of guidance for officials undertaking IAs. The use of IA tools in practice is highly differentiated, both between the main tool types (simpler tools tend to be more popular than more sophisticated ones) and amongst individual IA systems (tool use is generally higher amongst the older Member States than the newer ones). Therefore, the widespread institutionalisation of IA has not yet led to a concomitant institutionalisation of IA tool use. Indeed, many of the countries studied still appear unconvinced of the basic need to increase tool use across the board.   Impact Assessment tools are not perceived as a ‘silver bullet’ Some countries which have grappled with the challenge of how to increase IA tool use have done so more actively and firmly than others. But even amongst the most enthusiastic advocates, IA tools are not really seen as a ‘silver bullet’ to improve the quality of IA or, in turn, to produce ‘better’ policy decisions.   Impact Assessment tools need to be targeted towards user needs User needs with respect to IA tools defy simple generalisations: they tend to be specific to particular tools and/or IA systems. Instead of ‘saturating’ IA practitioners with information on tools, this pattern of use calls for a more targeted and ‘smarter’ deployment of existing as well as improved tools; one which is sensitive to the prevailing context in each country and policy case.       n.1 October 2012The Purpose and Context of Impact Assessment  The main purpose of Impact Assessment (IA) is not always made explicit in the underlying documents of the IA systems. In addition, an IA system can have more than one purpose and    opinion and can differ depending on who is asked.     ‘What are the purposes for doing IA in your country?’ (% of total responses)   Reducing costs imposed by regulation is an important driver for the introduction of IA across the majority of countries (e.g. Belgium, Cyprus, Poland and the UK).   Observing and following the implementation of IA in other jurisdictions (such as the lead given by the European Commission) also appears to be a factor in the introduction of some of the newer IA systems (e.g. Ireland, Greece and Poland). While sustainable development or the environment is mentioned in the IA Guidance in a number of countries (e.g. Finland, the Netherlands and the UK), this is seldom the main purpose of IA (see Figure 1).  The political and institutional context is important when examining IA systems, as the existing policy making process can have a strong influence on how IA is interpreted and practiced.          n.1 October 2012Impact Assessment Quality     A substantial proportion of the interviewees thought that the quality of IAs in their jurisdictions was still poor.        ‘What do you perceive the quality of the IAs in your country to be?’ (% of total responses)  Some countries have established mechanisms to ensure the adequacy and the quality of IAs. Most have units that provide guidance and coordination. However, formal quality control of the IA reports is established in only a few countries (e.g. the UK). Denmark, Finland, Cyprus and Greece have no central quality control. Increased quality control was one of the commonly suggested options by interviewees to improve the quality of IA. In countries where there is little central quality control it is not clear who is responsible for overseeing the IA system. Quality is left to the individual departments or even the policy officer in charge of the IA. Without proper scrutiny there appears to be little incentive to invest time and resources into IA.   Various factors are important in determining the quality of IAs. These include: the timing of the IA (i.e. is it done early, or late in the policy making process?); the level of political support for IA; the motivation of officials to conduct IAs; the level of skills (especially quantitative ones); the scope of the IAs (i.e. does it focus on the full range of impacts?). Political context can also be an important factor in the quality of IA and what is perceived as ‘quality’.         n.1 October 2012 The Purpose and Context of Impact Assessment    The main tools used in IA across the different countries are simple tools such as checklists and questionnaires, Cost Benefit Analysis (CBA) and Administrative Burden Assessments. Examples of other tools which are advocated and/or used less frequently include scenarios, Multi-Criteria Analysis, and computer models. The majority of the Guidance documents at least mention tools and some give in-depth instructions and/or worked examples. The Guidance documents in some countries act as simple tools themselves if they contain a number of checklists or are in the form of a questionnaire. Only a few countries do not advocate which tools should be used at all (e.g. Sweden; Switzerland). However, tool use is flexible in other countries (e.g. Italy; Denmark). Which tools are used therefore varies across and within countries with different departments favouring different tools.   In some countries, ministries are encouraged to develop tools for other ministries to apply. However, in other countries tool use is highly prescribed (e.g. the UK) and there is very little flexibility in which tools to apply or how to apply them. A number of countries also favour economic analysis (e.g. UK; Italy; Czech Republic; Netherlands; Belgium; Poland). Qualitative methods such as Multi-Criteria Analysis are only advocated in a few countries (e.g. Ireland). This is despite the fact that qualitative analysis is commonplace in IAs in practice. Similarly, quantification is less common than the guidelines would suggest and when it is done it is often incomplete or inadequate. This contributes to the mistrust felt by many policy officials towards quantitative tools.   There are opportunities to both increase the awareness of tools and to support their more extensive use. In some countries (particularly where tool use is more flexible) better awareness of the range of potential tools and what they can do is needed. In most countries, better training and support of officials is needed to adequately use tools, especially quantitative tools such as CBA. This goes hand in hand with providing adequate resources and time for tool use. The quality of tool use can also be improved by better guidance (including worked examples) as well as proper scrutiny of IAs.  In addition, there can be a strong cultural reluctance to engage with quantitative analysis. It is important to better understand the political, cultural, institutional factors which affect the appropriateness of certain tools (e.g. why certain apparently useful quantitative tools fail to get taken up).         n.1 October 2012                              The main purpose of the LIAISE Network of Excellence is to identify and exploit opportunities to bridge the existing gap between the research and the policy community in the field of Impact Assessment, improving the use of IA tools in policy making. LIAISE combines the multi-disciplinary competence of a core group of European research institutes.       The Policy Brief Series presents the results of the work carried out in LIAISE to the policy world. It addresses topics of current concern and focuses on those aspects of the issue where the policymaker (and the public opinion) is seeking additional information. 

Current practices and user expectations

https://refubium.fu-berlin.de/bitstream/fub188/17924/1/LIAISE_Policy_brief_01-1.pdf

Current practices and user expectations

Abstract

Similar works

Full text

Available Versions

Institutional Repository of the Freie Universität Berlin