Desirability–doability group judgment framework for the collaborative multicriteria evaluation of public policies

Desirability–doability framework (2 × D) is a novel framework for the collaborative evaluation of public policies. Fundamental objectives and performance indicators are agreed upon in workshops, policies are characterised, and barriers to implementation identified. MACBETH interactive protocols are then applied in decision conferences to elicit qualitative judgments about the desirability of policies, within and across objectives; and about their doability under the expected graveness of barriers on contrasting scenarios. Elicited judgments allow, respectively, to construct a shared multicriteria model measuring the overall desirability of policies; and, to measure their doability. Desirability–doability graphs enable visual interactive classification of policies, with sensitivity/robustness analyses of uncertainties. 2 × D was successfully tested in a real-world urban-health policymaking case to evaluate spatial policies. The main novelty of 2 × D is that it bridges the socio-technical gap, present in OR, between the support required by a complex social decision-making process, and that usually offered by analytic techniques – while keeping modeling theoretically sound and simple

Modelling multicriteria value interactions with Reasoning Maps

Idiographic causal maps are extensively employed in Operational Research to support problem structuring and complex decision making processes. They model means-end or causal discourses as a network of concepts connected by links denoting influence, thus enabling the representation of chains of arguments made by decision-makers. There have been proposals to employ such structures to support the structuring of multicriteria evaluation models, within an additive value measurement framework. However, a drawback of this multi-methodological modelling is the loss of richness of interactions along the means-end chains when evaluating options. This has led to the development of methods that make use of the structure of the map itself to evaluate options, such as the Reasoning Maps method, which employs ordinal scales and ordinal operators for such evaluation. However, despite their potential, Reasoning Maps cannot model explicitly value interactions nor perform a quantitative ranking of options, limiting their applicability and usefulness. In this article we propose extending the Reasoning Maps approach through a multilinear evaluation model structure, built with the MACBETH multicriteria method. The model explicitly captures the value interactions between concepts along the map and employs the MACBETH protocol of questioning to assess the strength of influence for each means-end link. The feasibility of the proposed approach to evaluate options and to deal with multicriteria interactions is tested in a real-world application to support the construction of a population health index

Pairwise comparison matrices and the error-free property of the decision maker

Pairwise comparison is a popular assessment method either for deriving criteria-weights or for evaluating alternatives according to a given criterion. In real-world applications consistency of the comparisons rarely happens: intransitivity can occur. The aim of the paper is to discuss the relationship between the consistency of the decision maker—described with the error-free property—and the consistency of the pairwise comparison matrix (PCM). The concept of error-free matrix is used to demonstrate that consistency of the PCM is not a sufficient condition of the error-free property of the decision maker. Informed and uninformed decision makers are defined. In the first stage of an assessment method a consistent or near-consistent matrix should be achieved: detecting, measuring and improving consistency are part of any procedure with both types of decision makers. In the second stage additional information are needed to reveal the decision maker’s real preferences. Interactive questioning procedures are recommended to reach that goal

Valuing health states: is the MACBETH approach useful for valuing EQ-5D-3L health states?

Background Quality Adjusted Life Years (QALYs) are a key outcome measure widely used within health technology assessment and health service research studies. QALYs combine quantity and quality of life, with quality of life calculations relying on the value of distinct health states. Such health states’ values capture the preferences of a population and have been typically built through numerical elicitation methods. Evidence points to these value scores being influenced by methods in use and individuals reporting cognitive difficulties in eliciting their preferences. Evidence from other areas has further suggested that individuals may prefer using distinct elicitation techniques and that this preference can be influenced by their numeracy. In this study we explore the use of the MACBETH (Measuring Attractiveness by a Categorical Based Evaluation Technique) non-numerical preference elicitation approach for health states’ evaluation. Methods A new protocol for preference elicitation based on MACBETH (only requiring qualitative judgments) was developed and tested within a web survey format. A sample of the Portuguese general population (n=243) valued 25 EQ-5D-3L health states with the MACBETH protocol and with a variant of the time trade-off (TTO) protocol, for comparison purposes and for understanding respondents’ preference for distinct protocols and differences in inconsistent evaluations. Respondents answered to a short numeracy test, and basic socio-economic information collected. Results Results show that the mean values derived from MACBETH and the TTO variant are strongly correlated; however, there are substantial differences for several health states’ values. Large and similar numbers of logical inconsistencies were found in respondents’ answers with both methods. Participants with higher levels of numeracy according to the test preferred expressing value judgments with MACBETH, while participants with lower levels were mostly indifferent to both methods. Higher correlations between MACBETH and TTO variant evaluations were observed for individuals with higher numeracy. Conclusion Results suggest that it is worth researching the use of non-numerical preference elicitation methods. Numeracy tests more appropriate for preference elicitation when no explicit considerations of uncertainty are made need to be explored and used. Further behavioural research is needed to fully understand the potential for using these methods in distinct settings (e.g. in different evaluation contexts and in face-to-face and non-face-to-face environments), as well as to explore the effect of literacy on assessments and on respondents’ preferences.UID/MULTI/4066/2016info:eu-repo/semantics/publishedVersio

Health-related quality of life and utilities in gastric premalignant conditions and malignant lesions: a multicentre study in a high prevalence country

BACKGROUND AND AIMS: A recent review of economic studies relating to gastric cancer revealed that authors use different tests to estimate utilities in patients with and without gastric cancer. Our aim was to determine the utilities of gastric premalignant conditions and adenocarcinoma with a single standardized health measure instrument. METHODS: Cross-sectional nationwide study of patients undergoing upper endoscopy (n=1,434) using the EQ-5D-5L quality of life (QoL) questionnaire. RESULTS: According to EQ-5D-5L, utilities in individuals without gastric lesions were 0.78 (95% confidence interval: 0.76-0.80), with gastric premalignant conditions 0.79 (0.77-0.81), previously treated for gastric cancer 0.77 (0.73-0.81) and with present cancer 0.68 (0.55-0.81). Self-reported QoL according to the visual analogue scale (VAS) for the same groups were 0.67 (0.66-0.69), 0.67 (0.66-0.69), 0.62 (0.59-0.65) and 0.62 (0.54-0.70) respectively. Utilities were consistently lower in women versus men (no lesions 0.71 vs. 0.78; premalignant conditions 0.70 vs. 0.82; treated for cancer 0.72 vs. 0.78 and present cancer 0.66 vs. 0.70). CONCLUSION: The health-related QoL utilities of patients with premalignant conditions are similar to those without gastric diseases whereas patients with present cancer show decreased utilities. Moreover, women had consistently lower utilities than men. These results confirm that the use of a single standardized instrument such as the EQ-5D-5L for all stages of the gastric carcinogenesis cascade is feasible and that it captures differences between conditions and gender dissimilarities, being relevant information for authors pretending to conduct further cost-utility analysis