Understanding Validity in Structuring Multi-Criteria Decision Problems

The first steps of multi-criteria decision making (MCDM) are typically the decomposition and structuring of the decision problem at hand. As all subsequent process steps of MCDM are based on the initial structuring of the decision problem, the validity of the structure representing the decision problem is of particular importance for the quality of the decision making process. This paper seeks to further develop our understanding of validity in structuring multi-criteria decisions. For this purpose, we link the structuring of decision problems in MCDM to the theory of chunking, which describes how human cognition structures and perceives environmental information. Based on this, we propose that the validity of models representing multi-criteria decision problems can be assessed by evaluating the degree to which they match the structures formed by chunking. We discuss a preliminary framework of how the match between the cognitive and the MCDM model can be tested. To demonstrate how this framework can be utilized in research practice, we apply it to empirically show that algorithmic, bottom-up structuring of MCDM problems leads to valid goal-criteria hierarchies

Pairwise comparison techniques for preference elicitation: using test-retest reliability as a quality indicator

This study aims at augmenting the profile distance method (PDM) with techniques that support the elicitation of a relative weight vector. Therefore, prominent methods from the field of preference elicitation and related research are investigated according to their task fitness. Preference elicitation is widely regarded as one integral part of decision support. It has found broad attention in multiple scientific fields, such as psychology of choice or behavioural decision making. The technique of pairwise comparison, often utilized in applied decision support, is examined according to practicability and validity for the estimation of a relative weight vector. The concrete task of weight vector assessment is tested in an experimental setting using widely accepted scales and techniques derived from the literature survey conducted. We distinctly identify two key figures for the measurement of outcome quality, accounting for both mathematical consistency and internal (or human) consistency. Preliminary experimental results from a web-based study with sixty two (62) distinct users provide for valuable insights in consistency ratio and test-retest reliability, indicating that unmodified pairwise comparisons are a suboptimal method for criteria preference elicitation. Additionally, we propose a number of improvements to practical preference elicitation, such as the use of a guided, process-based weight elicitation process

Evaluation of Techniques for Structuring Multi-Criteria Decision Problems

This article seeks to overcome the problem of structuring multi-criteria decision problems in a scientifically valid way. For this purpose, we theoretically and empirically compare two techniques which can be used for the purpose of structuring problem domains: card sorting procedures and statistical web mining. Based on two empirical test cases we assess whether decision structuring is reliable regarding the applied structuring method and whether the resulting hierarchies are valid representations of the decision problem at hand. The results indicate that the two techniques lead to quite different goal-criteria hierarchies and that web mining does not produce useful problem representations. In contrast, card sorting seems to be a valid structuring technique. We explain these results by the fact that card sorting procedures are interpretive techniques which are able to deal with vague concepts (criteria) while web mining, as a purely statistical approach, does not work well with ambiguous concepts

Explaining MCDM acceptance: a conceptual model of influencing factors

The number of newly developed Multi-Criteria Decision Making (MCDM) methods grew considerably in the last decades. Although their theoretical foundations are solid, there is still a lack of acceptance and application in the practical field. The objective of this research is the development of a conceptual model of factors that influence MCDM acceptance that serves as a starting point for further research. For this purpose, a broad diversified literature survey was conducted in the discipline of technology adoption and related topics (like human computer interaction) with special focus on MCDM acceptance. The constructs collected within the literature survey were classified based on a qualitative approach which yielded a conceptual model structuring the identified factors according to individual, social, technology-related, task-related and facilitating aspects. (author's abstract

A Semi-Automated Approach for Structuring Multi Criteria Decision Problems

This article seeks to enhance multi criteria decision making by providing a scientic approach for decomposing and structuring decision problems. We propose a process, based on concept mapping, which integrates group creativity techniques, card sorting procedures, quantitative data analysis and algorithmic automatization to construct meaningful and complete hierarchies of criteria. The algorithmic aspect is covered by a newly proposed recursive cluster algorithm, which automatically generates hierarchies from card sorting data. Based on comparison with another basic algorithm and empirical engineered and real-case test data, we validate that our process efficiently produces reasonable hierarchies of descriptive elements like goal- or problem-criteria. (authors' abstract