Methods to inform the development of concise objectives hierarchies in multi-criteria decision analysis

Building a well-structured objectives hierarchy is central to multi-criteria decision analysis (MCDA). However, in the absence of a systematic methodology to support the process, this task has been described as “more art than science”. Objectives hierarchies often tend to become large and constraining the size of a hierarchy can be challenging. This paper proposes and illustrates the use of a set of methods to support the simplification of the hierarchies in contexts that are “datarich” and characterised by many objectives. The aim of using the proposed approach is to support decision analysts in developing an appropriately concise decision model for the further interactions with the stakeholders. Using data from two completed environmental cases we show retrospectively how qualitative (means-ends networks), semiquantitative (relevancyanalysis) and quantitative (correlation analysis, principal component analysis, local sensitivity analysis of weights) methods, used alone or in combination, can inform hierarchy development. We evaluate the potential benefits and challenges of each method and discuss the advantages and disadvantages of the simplification of an objectives hierarchy. Questionnaire-based relevancy analysis can be a useful method to identify and communicate important objectives in the early phases of an MCDA process with stakeholders, while correlation analysis can help to identify overlapping objectives, particularly in cases having many objectives and alternatives. It is intended that the methods support a facilitator in developing a clear understanding of the problem and also prompt deeper thinking about and discussion of the appropriate structure and content of an objectives hierarchy with the stakeholders involved

Integrating multiple criteria decision analysis in participatory forest planning

Forest planning in a participatory context often involves multiple stakeholders with conflicting interests. A promising approach for handling these complex situations is to integrate participatory planning and multiple criteria decision analysis (MCDA). The objective of this paper is to analyze strengths and weaknesses of such an integrated approach, focusing on how the use of MCDA has influenced the participatory process. The paper outlines a model for a participatory MCDA process with five steps: stakeholder analysis, structuring of the decision problem, generation of alternatives, elicitation of preferences, and ranking of alternatives. This model was applied in a case study of a planning process for the urban forest in Lycksele, Sweden. In interviews with stakeholders, criteria for four different social groups were identified. Stakeholders also identified specific areas important to them and explained what activities the areas were used for and the forest management they wished for there. Existing forest data were combined with information from interviews to create a map in which the urban forest was divided into zones of different management classes. Three alternative strategic forest plans were produced based on the zonal map. The stakeholders stated their preferences individually by the Analytic Hierarchy Process in inquiry forms and a ranking of alternatives and consistency ratios were determined for each stakeholder. Rankings of alternatives were aggregated; first, for each social group using the arithmetic mean, and then an overall aggregated ranking was calculated from the group rankings using the weighted arithmetic mean. The participatory MCDA process in Lycksele is assessed against five social goals: incorporating public values into decisions, improving the substantive quality of decisions, resolving conflict among competing interests, building trust in institutions, and educating and informing the public. The results and assessment of the case study support the integration of participatory planning and MCDA as a viable option for handling complex forest-management situations. Key issues related to the MCDA methodology that need to be explored further were identified: 1) The handling of place-specific criteria, 2) development of alternatives, 3) the aggregation of individual preferences into a common preference, and 4) application and evaluation of the integrated approach in real case studies

Value-based assessment of new medical technologies: towards a robust methodological framework for the application of multiple criteria decision analysis in the context of health technology assessment

In recent years, multiple criteria decision analysis (MCDA) has emerged as a likely alternative to address shortcomings in health technology assessment (HTA) by offering a more holistic perspective to value assessment and acting as an alternative priority setting tool. In this paper, we argue that MCDA needs to subscribe to robust methodological processes related to the selection of objectives, criteria and attributes in order to be meaningful in the context of healthcare decision making and fulfil its role in value-based assessment (VBA). We propose a methodological process, based on multi-attribute value theory (MAVT) methods comprising five distinct phases, outline the stages involved in each phase and discuss their relevance in the HTA process. Importantly, criteria and attributes need to satisfy a set of desired properties, otherwise the outcome of the analysis can produce spurious results and misleading recommendations. Assuming the methodological process we propose is adhered to, the application of MCDA presents three very distinct advantages to decision makers in the context of HTA and VBA: first, it acts as an instrument for eliciting preferences on the performance of alternative options across a wider set of explicit criteria, leading to a more complete assessment of value; second, it allows the elicitation of preferences across the criteria themselves to reflect differences in their relative importance; and, third, the entire process of preference elicitation can be informed by direct stakeholder engagement, and can therefore reflect their own preferences. All features are fully transparent and facilitate decision making

Value-based assessment of new medical technologies: towards a robust methodological framework for the application of multiple criteria decision analysis in the context of health technology assessment

In recent years, multiple criteria decision analysis (MCDA) has emerged as a likely alternative to address shortcomings in health technology assessment (HTA) by offering a more holistic perspective to value assessment and acting as an alternative priority setting tool. In this paper, we argue that MCDA needs to subscribe to robust methodological processes related to the selection of objectives, criteria and attributes in order to be meaningful in the context of healthcare decision making and fulfil its role in value-based assessment (VBA). We propose a methodological process, based on multi-attribute value theory (MAVT) methods comprising five distinct phases, outline the stages involved in each phase and discuss their relevance in the HTA process. Importantly, criteria and attributes need to satisfy a set of desired properties, otherwise the outcome of the analysis can produce spurious results and misleading recommendations. Assuming the methodological process we propose is adhered to, the application of MCDA presents three very distinct advantages to decision makers in the context of HTA and VBA: first, it acts as an instrument for eliciting preferences on the performance of alternative options across a wider set of explicit criteria, leading to a more complete assessment of value; second, it allows the elicitation of preferences across the criteria themselves to reflect differences in their relative importance; and, third, the entire process of preference elicitation can be informed by direct stakeholder engagement, and can therefore reflect their own preferences. All features are fully transparent and facilitate decision making

Doctor of Philosophy

dissertationWith the growing national dissemination of the electronic health record (EHR), there are expectations that the public will benefit from biomedical research and discovery enabled by electronic health data. Clinical data are needed for many diseases and conditions to meet the demands of rapidly advancing genomic and proteomic research. Many biomedical research advancements require rapid access to clinical data as well as broad population coverage. A fundamental issue in the secondary use of clinical data for scientific research is the identification of study cohorts of individuals with a disease or medical condition of interest. The problem addressed in this work is the need for generalized, efficient methods to identify cohorts in the EHR for use in biomedical research. To approach this problem, an associative classification framework was designed with the goal of accurate and rapid identification of cases for biomedical research: (1) a set of exemplars for a given medical condition are presented to the framework, (2) a predictive rule set comprised of EHR attributes is generated by the framework, and (3) the rule set is applied to the EHR to identify additional patients that may have the specified condition. iv Based on this functionality, the approach was termed the ‘cohort amplification' framework. The development and evaluation of the cohort amplification framework are the subject of this dissertation. An overview of the framework design is presented. Improvements to some standard associative classification methods are described and validated. A qualitative evaluation of predictive rules to identify diabetes cases and a study of the accuracy of identification of asthma cases in the EHR using frameworkgenerated prediction rules are reported. The framework demonstrated accurate and reliable rules to identify diabetes and asthma cases in the EHR and contributed to methods for identification of biomedical research cohorts

Systematic Analysis of COVID-19 Ontologies

This comprehensive study conducts an in-depth analysis of existing COVID-19 ontologies, scrutinizing their objectives, classifications, design methodologies, and domain focal points. The study is conducted through a dual-stage approach, commencing with a systematic review of relevant literature and followed by an ontological assessment utilizing a parametric methodology. Through this meticulous process, twenty-four COVID-19 Ontologies (CovOs) are selected and examined. The findings highlight the scope, intended purpose, granularity of ontology, modularity, formalism, vocabulary reuse, and extent of domain coverage. The analysis reveals varying levels of formality in ontology development, a prevalent preference for utilizing OWL as the representational language, and diverse approaches to constructing class hierarchies within the models. Noteworthy is the recurrent reuse of ontologies like OBO models (CIDO, GO, etc.) alongside CODO. The METHONTOLOGY approach emerges as a favored design methodology, often coupled with application-based or data-centric evaluation methods. Our study provides valuable insights for the scientific community and COVID-19 ontology developers, supplemented by comprehensive ontology metrics. By meticulously evaluating and documenting COVID-19 information-driven ontological models, this research offers a comparative cross-domain perspective, shedding light on knowledge representation variations. The present study significantly enhances understanding of CovOs, serving as a consolidated resource for comparative analysis and future development, while also pinpointing research gaps and domain emphases, thereby guiding the trajectory of future ontological advancements.Comment: 16 pages, accepted for publication in 17th International Conference on Metadata and Semantics Research (MTSR2023), University of Milano-Bicocca, Milan, Italy, October 23-27, 202

Layered evaluation of interactive adaptive systems : framework and formative methods

Peer reviewedPostprin

A systems approach to evaluate One Health initiatives

Challenges calling for integrated approaches to health, such as the One Health (OH) approach, typically arise from the intertwined spheres of humans, animals, and ecosystems constituting their environment. Initiatives addressing such wicked problems commonly consist of complex structures and dynamics. As a result of the EU COST Action (TD 1404) “Network for Evaluation of One Health” (NEOH), we propose an evaluation framework anchored in systems theory to address the intrinsic complexity of OH initiatives and regard them as subsystems of the context within which they operate. Typically, they intend to influence a system with a view to improve human, animal, and environmental health. The NEOH evaluation framework consists of four overarching elements, namely: (1) the definition of the initiative and its context, (2) the description of the theory of change with an assessment of expected and unexpected outcomes, (3) the process evaluation of operational and supporting infrastructures (the “OH-ness”), and (4) an assessment of the association(s) between the process evaluation and the outcomes produced. It relies on a mixed methods approach by combining a descriptive and qualitative assessment with a semi-quantitative scoring for the evaluation of the degree and structural balance of “OH-ness” (summarised in an OH-index and OH-ratio, respectively) and conventional metrics for different outcomes in a multi-criteria-decision-analysis. Here, we focus on the methodology for Elements (1) and (3) including ready-to-use Microsoft Excel spreadsheets for the assessment of the “OH-ness”. We also provide an overview of Element (2), and refer to the NEOH handbook for further details, also regarding Element (4) (http://neoh.onehealthglobal.net). The presented approach helps researchers, practitioners, and evaluators to conceptualise and conduct evaluations of integrated approaches to health and facilitates comparison and learning across different OH activities thereby facilitating decisions on resource allocation. The application of the framework has been described in eight case studies in the same Frontiers research topic and provides first data on OH-index and OH-ratio, which is an important step towards their validation and the creation of a dataset for future benchmarking, and to demonstrate under which circumstances OH initiatives provide added value compared to disciplinary or conventional health initiatives

Influence of organisational culture on the implementation of health sector reforms in low and middle income countries : a qualitative interpretive review

The qualitative interpretive synthesis carried out for this MPH mini-dissertation reviews existing empirical literature for evidence on organisational culture and its influence on the implementation of health sector reforms in Low and Middle Income Countries. This mini-dissertation is organised into three parts: PART A: This is the review protocol which outlines the introduction, the background and the review questions for both the scoping review (which forms section B) and the qualitative interpretive synthesis ( which forms section C) along with their justifications. It also outlines the methodology for both the scoping review and the qualitative interpretive review. The literature search was carried out in eight electronic databases using key search terms developed from the review questions. Inclusion and exclusion criteria were developed to determine the articles for inclusion into the review. All the search terms, data extraction templates and summary tables used in both reviews are provided in this section. PART B: This is the literature review section which was carried out to map the scope of literature on organisational culture within the health sector in Low and Middle Income Countries in order to support the more detailed analysis in Section C. It begins with a general description of organisational culture and its conceptual frameworks, as well as a description of the tools used in assessing organisational culture that were identified from a broader reading of literature on organisational culture. The reviewer then describes the literature search strategy of the scoping review and maps the retrieved articles based on themes on organisational culture in the health sector. Lastly, the reviewer classifies the different dimensions of organisational culture identified in the reviewed articles using the Competing Values Framework in order to facilitate comparison of organisational culture across the studies. PART C: This is the full qualitative interpretive synthesis presented as a journal ready manuscript. This review begins with an introduction on health sector reforms and organisational culture. This is followed by a description of the methods used to identify the literature, an outline and synthesis of the findings, discussion section and lastly, the conclusion. The findings of this interpretive synthesis indicate the potential influence of various dimensions of organisational culture such as power distance, uncertainty avoidance, in-group and institutional collectivism, mediated through organisational practices, over the implementation of the health sector reforms. It also highlights the dearth of empirical literature around organisational culture and therefore, its results can only be tentative. There is need for health policy makers and health system researchers in Low and Middle Income Countries to conduct further analysis of organisational culture and change within the health system

Nature of the Evidence Base and Approaches to Guide Nutrition Interventions for Individuals: A Position Paper From the Academy of Nutrition Sciences

This Position Paper from the Academy of Nutrition Sciences is the third in a series which describe the nature of the scientific evidence and frameworks that underpin nutrition recommendations for health. This paper focuses on evidence which guides the application of dietary recommendations for individuals. In some situations, modified nutrient intake becomes essential to prevent deficiency, optimise development and health, or manage symptoms and disease progression. Disease and its treatment can also affect taste, appetite and ability to access and prepare foods, with associated financial impacts. Therefore, the practice of nutrition and dietetics must integrate and apply the sciences of food, nutrition, biology, physiology, behaviour, management, communication and society to achieve and maintain human health. Thus, there is huge complexity in delivering evidence-based nutrition interventions to individuals. This paper examines available frameworks for appraising the quality and certainty of nutrition research evidence, the development nutrition practice guidelines to support evidence implementation in practice and the influence of other sources of nutrition information and misinformation. The paper also considers major challenges in applying research evidence to an individual and suggests consensus recommendations to begin to address these challenges in the future. Our recommendations target three groups; those who deliver nutrition interventions to individuals, those funding, commissioning or undertaking research aimed at delivering evidence-based nutrition practice, and those disseminating nutritional information to individuals