Parameter-free aggregation of value functions from multiple experts and uncertainty assessment in multi-criteria evaluation

This paper makes a threefold contribution to spatial multi-criteria evaluation (MCE): firstly by presenting a new method concerning value functions, secondly by comparing different approaches to assess the uncertainty of a MCE outcome, and thirdly by presenting a case-study on land-use change. Even though MCE is a well-known methodology in GIScience, there is a lack of practicable approaches to incorporate the potentially diverse views of multiple experts in defining and standardizing the values used to implement input criteria. We propose a new method that allows generating and aggregating non-monotonic value functions, integrating the views of multiple experts. The new approach only requires the experts to provide up to four values, making it easy to be included in questionnaires. We applied the proposed method in a case study that uses MCE to assess the potential of future loss of vineyards in a wine-growing area in Switzerland, involving 13 experts from research, consultancy, government, and practice. To assess the uncertainty of the outcome three different approaches were used: firstly, a complete Monte Carlo simulation with the bootstrapped inputs, secondly a one-factor-at-a-time variation, and thirdly bootstrapping of the 13 inputs with subsequent analytical error propagation. The complete Monte Carlo simulation has shown the most detailed distribution of the uncertainty. However, all three methods indicate a general trend of areas with lower likelihood of future cultivation to show a higher degree of relative uncertainty

Parameter-free aggregation of value functions from multiple experts and uncertainty assessment in multi-criteria evaluation

This paper makes a threefold contribution to spatial multi-criteria evaluation (MCE): firstly by presenting a new method concerning value functions, secondly by comparing different approaches to assess the uncertainty of a MCE outcome, and thirdly by presenting a case-study on land-use change. Even though MCE is a well-known methodology in GIScience, there is a lack of practicable approaches to incorporate the potentially diverse views of multiple experts in defining and standardizing the values used to implement input criteria. We propose a new method that allows generating and aggregating non-monotonic value functions, integrating the views of multiple experts. The new approach only requires the experts to provide up to four values, making it easy to be included in questionnaires. We applied the proposed method in a case study that uses MCE to assess the potential of future loss of vineyards in a wine-growing area in Switzerland, involving 13 experts from research, consultancy, government, and practice. To assess the uncertainty of the outcome three different approaches were used: firstly, a complete Monte Carlo simulation with the bootstrapped inputs, secondly a one-factor-at-a-time variation, and thirdly bootstrapping of the 13 inputs with subsequent analytical error propagation. The complete Monte Carlo simulation has shown the most detailed distribution of the uncertainty. However, all three methods indicate a general trend of areas with lower likelihood of future cultivation to show a higher degree of relative uncertainty

A Multicriteria Approach for the Evaluation of the Sustainability of Re-use of Historic Buildings in Venice

The paper presents a multiple criteria model for the evaluation of the sustainability of projects for the economic re-use of historical buildings in Venice. The model utilises the relevant parameters for the appraisal of sustainability, aggregated into three macro-indicators: intrinsic sustainability, context sustainability and economic-financial feasibility. The model has been calibrated by a panel of experts and tested on two reuse hypotheses of the Old Arsenal in Venice. The tests have proven the model to be a useful support in the early stages of evaluation of re-use projects, where economic improvements are to be combined with conservation, as it supports the identification of critical points and the selection of projects, thus providing not only a check-list of variables to be considered, but an appraisal of trade-offs between economic uses and requirements of conservation.Economic Reuse, Historical Building Conservation

Managing Interacting Criteria: Application to Environmental Evaluation Practices

The need for organizations to evaluate their environmental practices has been recently increasing. This fact has led to the development of many approaches to appraise such practices. In this paper, a novel decision model to evaluate company’s environmental practices is proposed to improve traditional evaluation process in different facets. Firstly, different reviewers’ collectives related to the company’s activity are taken into account in the process to increase company internal efficiency and external legitimacy. Secondly, following the standard ISO 14031, two general categories of environmental performance indicators, management and operational, are considered. Thirdly, since the assumption of independence among environmental indicators is rarely verified in environmental context, an aggregation operator to bear in mind the relationship among such indicators in the evaluation results is proposed. Finally, this new model integrates quantitative and qualitative information with different scales using a multi-granular linguistic model that allows to adapt diverse evaluation scales according to appraisers’ knowledge

Development of an Aggregation Methodology for Risk Analysis in Aerospace Conceptual Vehicle Design

The growing complexity of technical systems has emphasized a need to gather as much information as possible regarding specific systems of interest in order to make robust, sound decisions about their design and deployment. Acquiring as much data as possible requires the use of empirical statistics, historical information and expert opinion. In much of the aerospace conceptual design environment, the lack of historical information and infeasibility of gathering empirical data relegates the data collection to expert opinion. The conceptual design of a space vehicle requires input from several disciplines (weights and sizing, operations, trajectory, etc.). In this multidisciplinary environment, the design variables are often not easily quantified and have a high degree of uncertainty associated with their values. Decision-makers must rely on expert assessments of the uncertainty associated with the design variables to evaluate the risk level of a conceptual design. Since multiple experts are often queried for their evaluation of uncertainty, a means to combine/aggregate multiple expert assessments must be developed. Providing decision-makers with a solitary assessment that captures the consensus of the multiple experts would greatly enhance the ability to evaluate risk associated with a conceptual design. The objective of this research has been to develop an aggregation methodology that efficiently combines the uncertainty assessments of multiple experts in multiple disciplines involved in aerospace conceptual design. Bayesian probability augmented by uncertainty modeling and expert calibration was employed in the methodology construction. Appropriate questionnaire techniques were used to acquire expert opinion; the responses served as input distributions to the aggregation algorithm. Application of the derived techniques were applied as part of a larger expert assessment elicitation and calibration study. Results of this research demonstrate that aggregation of uncertainty assessments in environments where likelihood functions and empirically assessed expert credibility factors are deficient is possible. Validation of the methodology provides evidence that decision-makers find the aggregated responses useful in formulating decision strategies

A heterogeneous multi-criteria multi-expert decision-support system for scoring combinations of flood mitigation and recovery options

In this study, we developed an innovative operational decision-support system (DSS) based on flood data and mitigation or recovery options, that can be used by both naïve and expert users to score portfolios of flood mitigation or recovery measures. The DSS combines exposure (i.e., economic, social, or environmental values at risk) and resilience (i.e., protection of the main equilibrium functions of human and physical systems). Experts from different fields define indices and functions, stakeholders express their attitudes towards risk, relative weights, and risk perceptions, and both groups use a shared learning process for risk assessment. The DSS algorithms include the "technique for order performance by similarity to ideal solution" (TOPSIS) and the "basic linguistic term set" (BLTS) methods for heterogeneous multi-criteria multi-expert decision-making. Decisions are illustrated using fixed or bounded values of flood depth, duration, and frequency, with plausible parameter values, for a case study of Cesenatico. The best mitigation option was construction of sand dunes and development of evacuation plans, which achieved 32% of the potential net benefit. The best recovery option was construction of sand dunes and development of evacuation plans and insurance schemes, which achieved 42% of the potential net benefit. Mitigation options outperformed recovery options whenever the relative importance of exposure with respect to resilience was greater than 95%. Sensitivity analysis revealed that the best mitigation option was most robust with respect to flood duration and depth; the best recovery option was most robust with respect to the relative weights attached to economic, social, and environmental factors. Both options were similarly robust with respect to interdependencies between the options

A Methodology for the Selection of Multi-Criteria Decision Analysis Methods in Real Estate and Land Management Processes

Real estate and land management are characterised by a complex, elaborate combination of technical, regulatory and governmental factors. In Europe, Public Administrators must address the complex decision-making problems that need to be resolved, while also acting in consideration of the expectations of the different stakeholders involved in settlement transformation. In complex situations (e.g., with different aspects to be considered and multilevel actors involved), decision-making processes are often used to solve multidisciplinary and multidimensional analyses, which support the choices of those who are making the decision. Multi-Criteria Decision Analysis (MCDA) methods are included among the examination and evaluation techniques considered useful by the European Community. Such analyses and techniques are performed using methods, which aim to reach a synthesis of the various forms of input data needed to define decision-making problems of a similar complexity. Thus, one or more of the conclusions reached allow for informed, well thought-out, strategic decisions. According to the technical literature on MCDA, numerous methods are applicable in different decision-making situations, however, advice for selecting the most appropriate for the specific field of application and problem have not been thoroughly investigated. In land and real estate management, numerous queries regarding evaluations often arise. In brief, the objective of this paper is to outline a procedure with which to select the method best suited to the specific queries of evaluation, which commonly arise while addressing decision-making problems. In particular issues of land and real estate management, representing the so-called “settlement sector”. The procedure will follow a theoretical-methodological approach by formulating a taxonomy of the endogenous and exogenous variables of the multi-criteria analysis method

Learning from Experts

The survey is concerned with the issue of information transmission from experts to non-experts. Two main approaches to the use of experts can be traced. According to the game-theoretic approach expertise is a case of asymmetric information between the expert, who is the better informed agent, and the non-expert, who is either a decision-maker or an evaluator of the expert’s performance. According to the Bayesian decision-theoretic approach the expert is the agent who announces his probabilistic opinion, and the non-expert has to incorporate that opinion into his beliefs in a consistent way, despite his poor understanding of the expert’s substantive knowledge. The two approaches ground the relationships between experts and non-experts on such different premises that their results are very poorly connected.Expert, Information Transmission, Learning