Managing stakeholder knowledge for the evaluation of innovation systems in the face of climate change

Purpose – The aim of this paper is to frame the stakeholder-driven system mapping approach in the context of climate change, building on stakeholder knowledge of system boundaries, key elements and interactions within a system and to introduce a decision support tool for managing and visualising this knowledge into insightful system maps with policy implications. Design/methodology/approach – This methodological framework is based on the concepts of market maps. The process of eliciting and visualising expert knowledge is facilitated by means of a reference implementation in MATLAB, which allows for designing technological innovation systems models in either a structured or a visual format. Findings – System mapping can contribute to evaluating systems for climate change by capturing knowledge of expert groups with regard to the dynamic interrelations between climate policy strategies and other system components, which may promote or hinder the desired transition to low carbon societies. Research limitations/implications – This study explores how system mapping addresses gaps in analytical tools and complements the systems of innovation framework. Knowledge elicitation, however, must be facilitated and build upon a structured framework such as technological innovation systems. Practical implications – This approach can provide policymakers with significant insight into the strengths and weaknesses of current policy frameworks based on tacit knowledge embedded in stakeholders. Social implications – The developed methodological framework aims to include societal groups in the climate policy-making process by acknowledging stakeholders’ role in developing transition pathways. The system map codifies stakeholder input in a structured and transparent manner. Originality/value – This is the first study that clearly defines the system mapping approach in the frame of climate policy and introduces the first dedicated software option for researchers and decision makers to use for implementing this methodology

Eliciting Expertise

Since the last edition of this book there have been rapid developments in the use and exploitation of formally elicited knowledge. Previously, (Shadbolt and Burton, 1995) the emphasis was on eliciting knowledge for the purpose of building expert or knowledge-based systems. These systems are computer programs intended to solve real-world problems, achieving the same level of accuracy as human experts. Knowledge engineering is the discipline that has evolved to support the whole process of specifying, developing and deploying knowledge-based systems (Schreiber et al., 2000) This chapter will discuss the problem of knowledge elicitation for knowledge intensive systems in general

Expert Elicitation for Reliable System Design

This paper reviews the role of expert judgement to support reliability assessments within the systems engineering design process. Generic design processes are described to give the context and a discussion is given about the nature of the reliability assessments required in the different systems engineering phases. It is argued that, as far as meeting reliability requirements is concerned, the whole design process is more akin to a statistical control process than to a straightforward statistical problem of assessing an unknown distribution. This leads to features of the expert judgement problem in the design context which are substantially different from those seen, for example, in risk assessment. In particular, the role of experts in problem structuring and in developing failure mitigation options is much more prominent, and there is a need to take into account the reliability potential for future mitigation measures downstream in the system life cycle. An overview is given of the stakeholders typically involved in large scale systems engineering design projects, and this is used to argue the need for methods that expose potential judgemental biases in order to generate analyses that can be said to provide rational consensus about uncertainties. Finally, a number of key points are developed with the aim of moving toward a framework that provides a holistic method for tracking reliability assessment through the design process.Comment: This paper commented in: [arXiv:0708.0285], [arXiv:0708.0287], [arXiv:0708.0288]. Rejoinder in [arXiv:0708.0293]. Published at http://dx.doi.org/10.1214/088342306000000510 in the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org

Evaluation of elicitation methods to quantify Bayes linear models

The Bayes linear methodology allows decision makers to express their subjective beliefs and adjust these beliefs as observations are made. It is similar in spirit to probabilistic Bayesian approaches, but differs as it uses expectation as its primitive. While substantial work has been carried out in Bayes linear analysis, both in terms of theory development and application, there is little published material on the elicitation of structured expert judgement to quantify models. This paper investigates different methods that could be used by analysts when creating an elicitation process. The theoretical underpinnings of the elicitation methods developed are explored and an evaluation of their use is presented. This work was motivated by, and is a precursor to, an industrial application of Bayes linear modelling of the reliability of defence systems. An illustrative example demonstrates how the methods can be used in practice

Dealing with abstraction: Case study generalisation as a method for eliciting design patterns

Developing a pattern language is a non-trivial problem. A critical requirement is a method to support pattern writers with abstraction, so as they can produce generalised patterns. In this paper, we address this issue by developing a structured process of generalisation. It is important that this process is initiated through engaging participants in identifying initial patterns, i.e. directly dealing with the 'cold-start' problem. We have found that short case study descriptions provide a productive 'way into' the process for participants. We reflect on a 1-year interdisciplinary pan-European research project involving the development of almost 30 cases and over 150 patterns. We provide example cases, detailing the process by which their associated patterns emerged. This was based on a foundation for generalisation from cases with common attributes. We discuss the merits of this approach and its implications for pattern development

Effective communication in requirements elicitation: A comparison of methodologies

The elicitation or communication of user requirements comprises an early and critical but highly error-prone stage in system development. Socially oriented methodologies provide more support for user involvement in design than the rigidity of more traditional methods, facilitating the degree of user-designer communication and the 'capture' of requirements. A more emergent and collaborative view of requirements elicitation and communication is required to encompass the user, contextual and organisational factors. From this accompanying literature in communication issues in requirements elicitation, a four-dimensional framework is outlined and used to appraise comparatively four different methodologies seeking to promote a closer working relationship between users and designers. The facilitation of communication between users and designers is subject to discussion of the ways in which communicative activities can be 'optimised' for successful requirements gathering, by making recommendations based on the four dimensions to provide fruitful considerations for system designers

Exploiting Qualitative Information for Decision Support in Scenario Analysis

The development of scenario analysis (SA) to assist decision makers and stakeholders has been growing over the last few years through mainly exploiting qualitative information provided by experts. In this study, we present SA based on the use of qualitative data for strategy planning. We discuss the potential of SA as a decision-support tool, and provide a structured approach for the interpretation of SA data, and an empirical validation of expert evaluations that can help to measure the consistency of the analysis. An application to a specific case study is provided, with reference to the European organic farming business

A knowledge-based decision support system for roofing materials selection and cost estimating: a conceptual framework and data modelling

A plethora of materials is available to the modern day house designer but selecting the appropriate material is a complex task. It requires synthesising a multitude of performance criteria such as initial cost, maintenance cost, thermal performance and sustainability among others. This research aims to develop a Knowledge-based Decision support System for Material Selection (KDSMS) that facilitates the selection of optimal material for different sub elements of a roof design. The proposed system also has a facility for estimating roof cost based on the identified criteria. This paper presents the data modelling conceptual framework for the proposed system. The roof sub elements are modelled on the Building Cost Information Service (BCIS) Standard Form of Cost Analysis. This model consists of a knowledge base and a database to store different types of roofing materials with their corresponding performance characteristics and rankings. The system s knowledge is elicited from an extensive review of literature and the use of a domain expert forum. The proposed system employs the multi criteria decision method of TOPSIS (Technique of ranking Preferences by Similarity to the Ideal Solution), to resolve the materials selection and optimisation problem. The KDSMS is currently being developed for the housing sector of Northern Ireland

Knowledge modeling for software design

This paper develops a modeling framework for systems engineering that encompasses systems modeling, task modeling, and knowledge modeling, and allows knowledge engineering and software engineering to be seen as part of a unified developmental process. This framework is used to evaluate what novel contributions the 'knowledge engineering' paradigm has made and how these impact software engineering