A Framework for Economics-Driven Assessment of Data Quality Decisions

Economic perspectives have raised growing attention in recent data quality (DQ) literature, as studies have associated DQ decisions with major cost-benefit tradeoffs. Despite the growing interest, DQ research has not yet developed a robust, agreedupon view for assessing and studying the link between DQ and economic outcome. As a contribution, this study proposes a framework, which links costs to the decisions made in managing the information process and improving the DQ, and benefits to the use of information-product outcomes by data consumers. Considering past research contributions, we develop this framework further into a high-level optimization model that permits quantitative assessment of cost-benefit tradeoffs, towards economically-optimal DQ decisions. We demonstrate a possible use of the proposed framework and the derived model, and highlight their potential contribution to an economics-driven view of DQ issues in both research and practice

SAFE Newsletter : 2013, Q3

Research: Joachim Weber, Benjamin Loos, Steffen Meyer, Andreas Hackethal "Individual Investors' Trading Motives and Security Selling Behavior" Ignazio Angeloni, Ester Faia "Monetary Policy and Prudential Regulations with Bank Runs" Helmut Siekmann "Legal Limits to Quantitative Easing" Policy Margit Vanberg "SAFE Summer Academy 2013 on 'International Financial Stability'" Guest Commentary Peter Praet "Cooperation between the ECB and Academia

Evaluating Software Architectures: Development Stability and Evolution

We survey seminal work on software architecture evaluationmethods. We then look at an emerging class of methodsthat explicates evaluating software architectures forstability and evolution. We define architectural stabilityand formulate the problem of evaluating software architecturesfor stability and evolution. We draw the attention onthe use of Architectures Description Languages (ADLs) forsupporting the evaluation of software architectures in generaland for architectural stability in specific

The utilisation of health research in policy-making: Concepts, examples, and methods of assessment

Chapter 1: Introduction and Background • The importance of utilising health research in policy-making, and therefore the need to understand the mechanisms involved, is increasingly recognised. Recent reports calling for more resources to improve health in developing countries, and global pressures for accountability, draw greater attention to research-informed policy-making. • For at least twenty years there has been recognition of the multiple meanings or models of research utilisation in policy-making. It has similarly been long recognised that a range of factors is involved in the interactions between health research and policy-makers. • The emerging focus on Health Research Systems (HRS) has identified additional mechanisms through which greater utilisation of research could be achieved. Assessment of the role of health research in policy-making is best undertaken as part of a wider study that also includes the utilisation of health research by industry, medical practitioners, and the public. Chapter 2: The Nature of Policy-Making, Types of Research and Utilisation Models • Policy-making broadly interpreted includes national health policies made by government ministers and officials, policies made by local health service managers, and clinical guidelines from professional bodies. In this report, however, the main focus is on public policy-making rather than that conducted by professional bodies. The utilisation of health research in policy-making should eventually lead to desired outcomes, including health gains. Research can make a contribution in at least three phases of the policy-making process: agenda setting; policy formulation; and implementation. Descriptions of these processes, however, can over-estimate the degree of rationality in policy-making. Therefore, the analysis is informed by a review of the full range of policy-making models. These include rational and incrementalist models. • Various categories of research are likely to be used differently in health policy-making. Applied research might be more readily useable by a policy system than basic research, but health policy-makers tend to relate more willingly to natural sciences than social sciences. When research is based on the priorities of potential users, and/or is research of proven quality, this increases the possibility that it will be translated into policies. There also appears to be a greater chance of research being used in clinical policies about delivering care to patients, than in national policies on the structures of the health service. • Models of research utilisation in policy-making start with a link to rational or instrumental views of policy-making, and include descriptions of how commissioned research can help to find solutions to problems. Other models relate to an incrementalist view in which policy-making involves a series of small steps over a long period; research findings might gradually cause a shift in perceptions about an issue in a process of ‘enlightenment’. Interactive models of research utilisation stress the way in which policy-makers and researchers might develop links over a long period. Research can also be used symbolically to support decisions already taken. Chapter 3: Examples from Previous Studies • A study of health policy-making in two southern African countries illustrates how policy-making processes can be analysed. It addresses agenda setting, policy formulation and implementation. The methods used included documentary analysis and key informant interviews. • Many previous studies of research utilisation can provide lessons for future assessments. Two broad approaches can be identified. Some studies start with pieces, or programmes, of research and examine their impact. Others consider policy on a particular topic and assess the role of research in the policy-making. There are advantages and drawbacks in each approach, and overlaps between them. • To facilitate comparison, studies of research utilisation are best organised around a conceptual framework. Despite that, the influence of contextual factors in different settings makes it difficult to generalise. • The two methods used most frequently, and usually together, come from the qualitative tradition: documentary analysis and in-depth interviews. Questionnaires, bibliometric analysis, insider knowledge and historical approaches have all been applied. A few recent studies have attempted to score or scale the level of utilisation. • The examples suggest there is a greater level of utilisation and final outcomes in terms of health, health equity, and social and economic gain than is often assumed, whilst still showing much underutilisation. There is considerable variation in the degree of utilisation, both within and between studies. Chapter 4: Key Issues in the Analysis of Research Utilisation in Policy-Making • Increasing attention is focusing on the concept of interfaces between researchers and the users of research. This incorporates the idea that there are likely to be different values and interests between the two communities. • In relation to utilisation, the prioritisation debate revolves around two key aspects: whether priorities are being set that will produce research that policy-makers and others will want to use, and whether priorities are being set that will engage the interests and commitment of the research community. • Interactions across the interface between policy-makers and researchers are important in transferring research to policy-makers. This fits especially well with the interactive model of utilisation. Actions by individual researchers can be useful in generating interaction, but it is desirable to consider the role of the HRS in encouraging or facilitating interactions, networks and mechanisms at a system-wide level. The HRS could provide funding and organisational support for various items including: long-term research centres; research brokerage/translator mechanisms; the creation of official committees of policy-makers and researchers; and mechanisms for review and synthesis of research findings. • There is increased recognition of the significance of policy-makers in their role as the receptors of research. In relation to the perspective of policy-makers there is a spectrum of key questions. These range from whether relevant research is available and effectively being brought to their attention, to whether they are able to absorb it and willing to use it. The HRS has a responsibility, especially in the early parts of the spectrum, but the wider health system also has a responsibility to create appropriate institutional mechanisms and ensure there are staff willing and able to incorporate relevant research. • More attention should be given to the role of incentives, both for researchers to produce utilisable research, and for policy-makers, at the system or individual level, to use it. The assessment of utilisation becomes a key issue if rewards are to focus on relevance as well as research excellence. • An appropriate model for assessing research utilisation in policy-making combines analysis of two issues: the role of receptors and the importance of actions at the interfaces. An emphasis on the role of the receptor is necessary because ultimately it is up to the policy-maker to make the decisions. Any assessment of the success of the HRS in relation to utilisation must accept that the wider political context is beyond the control of the HRS, but consider the activities of the HRS, within its given context, to enhance the utilisation of research by increasing the permeability of the interfaces. Chapter 5: Assessment of Research Utilisation in Health Policy-Making • The reasons for assessing the utilisation of research in policy-making include: advocacy, accountability, and increased understanding. For the World Health Organization there could be a role in conducting such assessments with the aim of providing evidence of the effective use of research resources. This could support advocacy for greater resources to be made available for health research. It is important that the purposes of any assessment are taken into account in planning the methods to be used. • Previous studies demonstrated the difficulties of making generalisations about specific factors associated with high levels of utilisation. To address this in any cross-national WHO initiative involving a series of studies in a range of countries, it would be desirable to structure all the studies around a conceptual framework (such as the interfaces and receptor framework considered here) and base the studies in each country on common themes. These could include policies for the adoption of multi-drug therapy for treating leprosy, and for the equitable access to health services. • Analysis of documents and semi-structured interviews would be appropriate methods in each study assessing the role of research in policy-making on a specific policy theme. Questionnaires could also have a role. These approaches would provide triangulation of methods and data-sources and should also provide material to help identify the relative importance, in relation to the level of utilisation recorded, of the HRS mechanisms described in the previous analysis. The types and sources of research used, and reasons for their use, should also be recorded and attempts made to correlate them with the previous priority setting approaches. It is expected that each study will produce its own narrative or story of what caused utilisation in the particular context, but the data gathered could also be applied to descriptive scales of the level research utilisation. The four scales could cover the consistency of policy with research findings, and the degree of influence of research on agenda setting, policy formulation, and implementation. • The findings from the assessments in each participating country should be collated. For each policy theme or topic the analysis would compare two sets of data: the scales for level of research utilisation in each country, and the contextualised lists of the HRS activities and other mechanisms and networks thought to be important. Although the account here has focused on research impact on policy-making, the evaluations would be stronger as part of a wider analysis covering research utilisation and interactions with practitioners, industry and the public. • Given appropriate and targeted topic and country selection, this approach is likely to meet the purpose of using structured methods to provide examples of effective research utilisation. The approach should contribute towards enhanced understanding of the issues and could provide the basis of an assessment tool which, if used widely in countries, could lead to greater utilisation of health research.Research Policy and Co-operation (RPC) Department of the World Health Organization, Geneva; UK Department of Health’s Policy Research Programme; Alliance for Health Policy and Systems Research from the governments of Norway and Sweden; World Bank and International Development Research Council of Canad

Using megaproject performance outcomes to enhance decision-making behaviours in civil engineering graduates

A comprehensive review of megaproject performance propositioned the issues leading to project failure as behavioral. Project failure linked cost overruns and schedule delays to acts of delusion and deception, citing the ability to learn lessons, and the misalignment of incentives as influencing factors. A mixed-methods study was designed to gain insight into the decision-making behaviors of undergraduate civil engineers, and the role that education could play in enhancing decision-making. A co-curricular intervention led to qualitative exploration of decision-making in civil engineering undergraduates. Motivation featured heavily, particularly a conflict between interest and enjoyment, and the reward structures of traditional education and industry. Results led to a quantitative measure of intrinsic motivation and critical thinking ability. Findings from the educational environment have implications for industry and led to recommendations regarding the importance of autonomy and relatedness, to a megaproject environment

Reconfiguring Household Management in Times of Discontinuity as an Open System: The Case of Agro-food Chains

The file attached to this record is the author's final peer reviewed version. The Publisher's final version can be found by following the DOI link.This article is based upon a heterodox approach to economics that rejects the oversimplification made by closed economic models and the mainstream concept of ‘externality.’ This approach re-imagines economics as a holistic evaluation of resources versus human needs, which requires judgement based on understanding of the complexity generated by the dynamic relations between different systems. One re-imagining of the economic model is as a holistic and systemic evaluation of agri-food systems’ sustainability that was performed through the multi-dimensional Governance Assessment Matrix Exercise (GAME). This is based on the five capitals model of sustainability, and the translation of qualitative evaluations into quantitative scores. This is based on the triangulation of big data from a variety of sources. To represent quantitative interactions, this article proposes a provisional translation of GAME’s qualitative evaluation into a quantitative form through the identification of measurement units that can reflect the different capital dimensions. For instance, a post-normal, ecological accounting method, Emergy is proposed to evaluate the natural capital. The revised GAME re-imagines economics not as the ‘dismal science,’ but as one that has potential leverage for positive, adaptive and sustainable ecosystemic analyses and global ‘household’ management. This article proposes an explicit recognition of economics nested within the social spheres of human and social capital which are in turn nested within the ecological capital upon which all life rests and is truly the bottom line. In this article, the authors make reference to an on-line retailer of local food and drink to illustrate the methods for evaluation of the five capitals model

The non-use and influence of UK energy sector indicators

This paper presents the results from a case study on the role in policymaking of UK Energy Sector Indicators (ESIs), introduced by the government in 2003. The findings show that the ESIs constituted a very minor element within the broader evidence-base used by policymakers, and that this indicator set and its objectives were poorly known even to central players in the sector. The findings of this research provide further evidence for the observation that scientific knowledge (including evaluations, assessments and indicators) seldom play an instrumental role in policymaking, and are more likely to produce indirect, conceptual and political impacts. The analysis provides a number of tentative conclusions concerning such potential indirect impacts that accrue mainly through processes of dialogue and argumentation both during the preparation of the indicators and after their publication as part of the annual reporting by the UK energy department. The ESIs have played various conceptual and political roles, yet the concrete outcomes in terms of policy change remain to be explored. The conclusions highlight the limitations of rationalist notions of direct, instrumental use in the efforts to understand the role of indicators in policymaking. The paper concludes by three tentative propositions concerning the explanations to the absence of instrumental role of the ESIs, which could be usefully explored in future research: the characteristics of the energy sector; the characteristics of the UK policy culture; and the exceptionality of the ESIs in the general evidence-base of UK energy sector

Solving multiple-criteria R&D project selection problems with a data-driven evidential reasoning rule

In this paper, a likelihood based evidence acquisition approach is proposed to acquire evidence from experts'assessments as recorded in historical datasets. Then a data-driven evidential reasoning rule based model is introduced to R&D project selection process by combining multiple pieces of evidence with different weights and reliabilities. As a result, the total belief degrees and the overall performance can be generated for ranking and selecting projects. Finally, a case study on the R&D project selection for the National Science Foundation of China is conducted to show the effectiveness of the proposed model. The data-driven evidential reasoning rule based model for project evaluation and selection (1) utilizes experimental data to represent experts' assessments by using belief distributions over the set of final funding outcomes, and through this historic statistics it helps experts and applicants to understand the funding probability to a given assessment grade, (2) implies the mapping relationships between the evaluation grades and the final funding outcomes by using historical data, and (3) provides a way to make fair decisions by taking experts' reliabilities into account. In the data-driven evidential reasoning rule based model, experts play different roles in accordance with their reliabilities which are determined by their previous review track records, and the selection process is made interpretable and fairer. The newly proposed model reduces the time-consuming panel review work for both managers and experts, and significantly improves the efficiency and quality of project selection process. Although the model is demonstrated for project selection in the NSFC, it can be generalized to other funding agencies or industries.Comment: 20 pages, forthcoming in International Journal of Project Management (2019

The utilisation of health research in policy-making: Concepts, examples and methods of assessment

The importance of health research utilisation in policy-making, and of understanding the mechanisms involved, is increasingly recognised. Recent reports calling for more resources to improve health in developing countries, and global pressures for accountability, draw greater attention to research-informed policy-making. Key utilisation issues have been described for at least twenty years, but the growing focus on health research systems creates additional dimensions. The utilisation of health research in policy-making should contribute to policies that may eventually lead to desired outcomes, including health gains. In this article, exploration of these issues is combined with a review of various forms of policy-making. When this is linked to analysis of different types of health research, it assists in building a comprehensive account of the diverse meanings of research utilisation. Previous studies report methods and conceptual frameworks that have been applied, if with varying degrees of success, to record utilisation in policy-making. These studies reveal various examples of research impact within a general picture of underutilisation. Factors potentially enhancing utilisation can be identified by exploration of: priority setting; activities of the health research system at the interface between research and policy-making; and the role of the recipients, or 'receptors', of health research. An interfaces and receptors model provides a framework for analysis. Recommendations about possible methods for assessing health research utilisation follow identification of the purposes of such assessments. Our conclusion is that research utilisation can be better understood, and enhanced, by developing assessment methods informed by conceptual analysis and review of previous studies