Balancing costs and benefits at different stages of medical innovation: a systematic review of Multi-criteria decision analysis (MCDA)

Background The diffusion of health technologies from translational research to reimbursement depends on several factors included the results of health economic analysis. Recent research identified several flaws in health economic concepts. Additionally, the heterogeneous viewpoints of participating stakeholders are rarely systematically addressed in current decision-making. Multi-criteria Decision Analysis (MCDA) provides an opportunity to tackle these issues. The objective of this study was to review applications of MCDA methods in decisions addressing the trade-off between costs and benefits. Methods Using basic steps of the PRISMA guidelines, a systematic review of the healthcare literature was performed to identify original research articles from January 1990 to April 2014. Medline, PubMed, Springer Link and specific journals were searched. Using predefined categories, bibliographic records were systematically extracted regarding the type of policy applications, MCDA methodology, criteria used and their definitions. Results 22 studies were included in the analysis. 15 studies (68 %) used direct MCDA approaches and seven studies (32 %) used preference elicitation approaches. Four studies (19 %) focused on technologies in the early innovation process. The majority (18 studies - 81 %) examined reimbursement decisions. Decision criteria used in studies were obtained from the literature research and context-specific studies, expert opinions, and group discussions. The number of criteria ranged between three up to 15. The most frequently used criteria were health outcomes (73 %), disease impact (59 %), and implementation of the intervention (40 %). Economic criteria included cost-effectiveness criteria (14 studies, 64 %), and total costs/budget impact of an intervention (eight studies, 36 %). The process of including economic aspects is very different among studies. Some studies directly compare costs with other criteria while some include economic consideration in a second step. Conclusions In early innovation processes, MCDA can provide information about stakeholder preferences as well as evidence needs in further development. However, only a minority of these studies include economic features due to the limited evidence. The most important economic criterion cost-effectiveness should not be included from a technical perspective as it is already a composite of costs and benefit. There is a significant lack of consensus in methodology employed by the various studies which highlights the need for guidance on application of MCDA at specific phases of an innovation

Why is it so difficult to integrate ethics in Health Technology Assessment (HTA)? The epistemological viewpoint

Ethics has been identified as a key element in Health Technology Assessment (HTA) since its conception. However, ethical issues are still not frequently addressed explicitly in HTA. Several valuable reasons have been identified. The basis of the article is the claim that ethics is often not part of HTA for “epistemological reasons”. Hence, the main aim of the contribution is to explore in more details and emphasize them by using the fact/value dichotomy. Our conclusion is that current HTA configuration is dominantly based on the comparison among objective and empirically testable “facts”, whilst ethics is not empirically testable. In this sense, there is a sort of “epistemological gap”, which can explain why it is so difficult to integrate ethics in HTA. We suggest that the epistemological differences among the various domains of HTA are addressed more explicitly

Guidance on the integrated assessment of complex health technologies: the INTEGRATE-HTA model

Challenges in assessments of health technologies In recent years there have been major advances in the development of health technology assessment (HTA). However, HTA still has certain limitations when assessing technologies which are complex, i.e. consist of several interacting components, target different groups or organizational levels, have multiple and variable outcomes, and/or permit a certain degree of flexibility or tailoring (Craig et al., 2008), fi are context-dependent - current HTA usually focusses on the technology, not on the system within which it is used, fi perform differently depending on the way they are implemented, fi have different effects on different individuals. Furthermore, HTA usually assesses and appraises aspects side-by-side, while decision-making needs an integrated perspective on the value of a technology. In the EU-funded INTEGRATE-HTA project, we developed concepts and methods to deal with these challenges, which are described in six guidances. Because of the interactions, an integrated assessment needs to start from the beginning of the assessment. This guidance provides a systematic five-step-process for an integrated assessment of complex technologies (the INTEGRATE-HTA Model). Purpose and scope of the guidance The aim of the INTEGRATE-HTA project is to provide concepts and methods that enable a patient-centred, comprehensive, and integrated assessment of complex health technologies. The purpose of this guidance is to structure the overall HTA-process. The INTEGRATE-HTA Model outlines an integrated scoping process, a coordinated application of assessment methods for different aspects and an integrated and structured decision-making process. It is intended for HTA agencies, HTA researchers and those engaged in the evaluation of complex health technologies. As it links the assessment to the decision-making process, it also addresses HTA commissioners and other stakeholders using or planning HTAs. While all technologies are arguably complex, some are more complex than others. Applying this guidance might lead to a more thorough and therefore more time-consuming process. Depending on the degree of complexity, one might choose to follow the whole process as described in this guidance, or only focus on certain steps. The guidance provides an operational definition to assess the complexity of technologies which can be used to identify specific aspects that will need more attention than others. What the guidance does not provide is a post-hoc solution for assessments that have already been completed. | 6 Development of the guidance The INTEGRATE-HTA Model presented in this guidance was developed based on a systematic literature search on approaches for integration, on the experiences of traditional HTAs, as well as on the other methodological guidances developed in the INTEGRATE-HTA project. It was tested in a case study on palliative care and iteratively revised during the practical application. The guidance was again revised after internal and external peer-review. Application of this guidance For a comprehensive integrated assessment of a complex technology, we developed a five-step process, the INTEGRATE-HTA model. In Step 1, the HTA objective and the technology are defined with the support from a panel of stakeholders. An initial logic model is developed in Step 2. The initial logic model provides a structured overview of the technology, the context, implementation issues, and relevant patient groups. It then frames the assessment of the effectiveness, as well as economic, ethical, legal, and socio-cultural aspects in Step 3. In Step 4, a graphical overview of the assessment results, structured by the logic model, is provided. Step 5 is a structured decision-making process informed by the HTA (and is thus not formally part of the HTA, but follows it). fi Step 1: In step 1, the technology under assessment and the objective of the HTA are defined. Especially for complex technologies, such as palliative care, the definition of the technology alone is a challenge that must not be underestimated. It is recommended to do this based on a tentative literature review and with the support of stakeholder advisory panels (SAPs) which should comprise clinical experts, academics, patients, possibly their relatives and/or other caretakers, and the public. The setting of an objective considering all relevant aspects of complexity and structured by assessment criteria is important. The assessment criteria will usually reflect values of the stakeholders as well as the input from the theoretical, methodological and empirical literature. fi Step 2: In step 2, an initial logic model is developed (see Guidance on the use of logic models in health technology assessments of complex interventions). The model provides a structured overview on participants, interventions, comparators, and outcomes. Parallel to this, groups of patients that are distinguished by different preferences and treatment moderators (see Guidance for the assessment of treatment moderation and patients’ preferences) are identified. Specific context and implementation issues are also identified as part of the initial logic model (see Guidance for the Assessment of Context and Implementation in Health Technology Assessments (HTA) and Systematic Reviews of Complex Interventions). The product of this step is the logic model as a graphical representation of all aspects and their interactions that are relevant for the assessment of the complex technology. fi Step 3: In step 3, the logic model serves as a conceptual framework that guides the evidence assessment. Depending on the specific aspect (e.g. effectiveness, economic, ethical, socio-cultural, or legal aspects) different methods are available for the assessment (see Guidance for assessing effectiveness, economic aspects, ethical aspects, socio-cultural aspects and legal aspects in complex technologies). The outputs of step 3 are evidence reports and standardized evidence summaries for each assessment aspect (e.g. report on economics, report on ethical aspects, etc.). fi Step 4: In step 4, the assessment results of step 3 are structured using the logic model developed in step 2. Whereas the initial logic model in step 2 specifies what evidence is relevant, the extended logic model to assist decision-making in step 4 visualizes the assessment results as well as the interaction with respect to the HTA objectives. It also allows for the consideration of different scenarios depending on the variation in context, implementation and patient characteristics. 7 | fi Step 5: Step 5 involves a structured decision-making process and is not an integral part of the HTA in the narrow sense. Decision-making can be supported by applying quantitative e.g. MCDA- (Multi-criteria decision analysis) or qualitative decision support tools. Flexibility in the application of these tools by the decision committee is crucial, taking different decision settings and evidence needs into consideration. Conclusions In current HTA, different aspects are usually assessed and presented independent of each other. Context, implementation issues and patient characteristics are rarely considered. The INTEGRATE-HTA Model enables a coordinated assessment of all these aspects and addresses their interdependencies. The perspective of stakeholders such as patients and professionals with their values and preferences is integrated in the INTEGRATE-HTA Model to obtain HTA results that are meaningful for all relevant stakeholders. Finally, health policy makers obtain an integrated perspective of the assessment results to achieve fair and legitimate conclusions at the end of the HTA process. The application of the model will usually require more time and resources than traditional HTA. An initial assessment of the degree and the character of complexity of a technology might be helpful to decide whether or not the whole process or only specific elements will be applied

Stakeholder involvement throughout health technology assessment: an example from palliative care

Objectives: Internationally, funders require stakeholder involvement throughout health technology assessment (HTA). We report successes, challenges, and lessons learned from extensive stakeholder involvement throughout a palliative care case study that demonstrates new concepts and methods for HTA. Methods: A 5-step “INTEGRATE-HTA Model” developed within the INTEGRATE-HTA project guided the case study. Using convenience or purposive sampling or directly / indirectly identifying and approaching individuals / groups, stakeholders participated in qualitative research or consultation meetings. During scoping, 132 stakeholders, aged ≥ 18 years in seven countries (England, Italy, Germany, The Netherlands, Norway, Lithuania, and Poland), highlighted key issues in palliative care that assisted identification of the intervention and comparator. Subsequently stakeholders in four countries participated in face–face, telephone and / or video Skype meetings to inform evidence collection and / or review assessment results. An applicability assessment to identify contextual and implementation barriers and enablers for the case study findings involved twelve professionals in the three countries. Finally, thirteen stakeholders participated in a mock decision-making meeting in England. Results: Views about the best methods of stakeholder involvement vary internationally. Stakeholders make valuable contributions in all stages of HTA; assisting decision making about interventions, comparators, research questions; providing evidence and insights into findings, gap analyses and applicability assessments. Key challenges exist regarding inclusivity, time, and resource use. Conclusion: Stakeholder involvement is feasible and worthwhile throughout HTA, sometimes providing unique insights. Various methods can be used to include stakeholders, although challenges exist. Recognition of stakeholder expertise and further guidance about stakeholder consultation methods is needed

A systematic review of Health Technology Assessment tools in sub-Saharan Africa: methodological issues and implications

Background Health technology assessment (HTA) is mostly used in the context of high- and middle-income countries. Many “resource-poor” settings, which have the greatest need for critical assessment of health technology, have a limited basis for making evidence-based choices. This can lead to inappropriate use of technologies, a problem that could be addressed by HTA that enables the efficient use of resources, which is especially crucial in such settings. There is a lack of clarity about which HTA tools should be used in these settings. This research aims to provide an overview of proposed HTA tools for “resource-poor” settings with a specific focus on sub-Saharan Africa (SSA). Methodology A systematic review was conducted using basic steps from the PRISMA guidelines. Studies that described HTA tools applicable for “resource-limited” settings were identified and critically appraised. Only papers published between 2003 and 2013 were included. The identified tools were assessed according to a checklist with methodological criteria. Results Six appropriate tools that are applicable in the SSA setting and cover methodological robustness and ease of use were included in the review. Several tools fulfil these criteria, such as the KNOW ESSENTIALS tool, Mini-HTA tool, and Multi-Criteria Decision Analysis but their application in the SSA context remains limited. The WHO CHOICE method is a standardized decision making tool for choosing interventions but is limited to their cost-effectiveness. Most evaluation of health technology in SSA focuses on priority setting. There is a lack of HTA tools that can be used for the systematic assessment of technology in the SSA context. Conclusions An appropriate HTA tool for “resource-constrained” settings, and especially SSA, should address all important criteria of decision making. By combining the two most promising tools, KNOW ESSENTIALS and Multi-Criteria Decision Analysis, appropriate analysis of evidence with a robust and flexible methodology could be applied for the SSA setting

Integrated assessment of home based palliative care with and without reinforced caregiver support: ‘A demonstration of INTEGRATE-HTA methodological guidances’

About this Executive Summary This comprehensive executive summary reports on a case study designed to demonstrate the application of a number of the key concepts and methods developed in the INTEGRATE-HTA project to the assessment of complex technologies. The case study focuses on models of home based palliative care with and without an additional element of caregiver support, known as reinforced and non-reinforced home based palliative care respectively. What is already known about the topic With changing disease patterns in Europe, increasingly complex health care technologies, such as palliative care, have gained importance. Current HTA methods rarely take account of wider legal, ethical and socio-cultural issues or context and implementation and are not adequately equipped to assess highly complex technologies, despite considerable progress in recent years. As a result, HTA is rarely applied to highly complex health technologies. What this case study report adds The INTEGRATE-HTA project developed concepts and methods for the assessment of complex technologies, taking into account legal, ethical and socio-cultural issues as well as context and implementation. This case study is designed to demonstrate the application of a number of the key concepts and methods developed in the INTEGRATE-HTA project to home based palliative care, with and without an additional element of caregiver support, as an example of a complex technology; known as “reinforced” and “non-reinforced” home based palliative care respectively. The INTEGRATE-HTA model, developed to enable integration of relevant assessment aspects, is used to structure this report. The case study reports on the application of some of the concepts and methods developed within the INTEGRATE-HTA project to the assessment of effectiveness as well as economic, sociocultural, ethical, and legal aspects; patient preferences and patient-specific moderators of treatment and context and implementation issues related to reinforced and non-reinforced home based palliative care. The highly complex nature of reinforced and non-reinforced home based palliative care is illustrated through an assessment of complexity characteristics. | 4 The case study also involved extensive lay and professional stakeholder involvement, using a variety of methods, to inform the HTA process at key stages throughout the project. Key messages are highlighted for both HTA research and for palliative care. The implications for research and practice The concepts and methods developed in the INTEGRATE-HTA project have been shown to be feasible and to have the potential to offer added value, but require further development and application in the assessment of other complex technologies. Reinforced and non-reinforced models palliative care are highly complex. The case study findings offer some insights into their effectiveness as well as economic, sociocultural, ethical, and legal issues; patient preferences and patient-specific moderators of treatment as well as context and implementation issues

Exploring the perspectives and preferences for HTA across German healthcare stakeholders using a multi-criteria assessment of a pulmonary heart sensor as a case study

Background Health technology assessment and healthcare decision-making are based on multiple criteria and evidence, and heterogeneous opinions of participating stakeholders. Multi-criteria decision analysis (MCDA) offers a potential framework to systematize this process and take different perspectives into account. The objectives of this study were to explore perspectives and preferences across German stakeholders when appraising healthcare interventions, using multi-criteria assessment of a heart pulmonary sensor as a case study. Methods An online survey of 100 German healthcare stakeholders was conducted using a comprehensive MCDA framework (EVIDEM V2.2). Participants were asked to provide i) relative weights for each criterion of the framework; ii) performance scores for a health pulmonary sensor, based on available data synthesized for each criterion; and iii) qualitative feedback on the consideration of contextual criteria. Normalized weights and scores were combined using a linear model to calculate a value estimate across different stakeholders. Differences across types of stakeholders were explored. Results The survey was completed by 54 participants. The most important criteria were efficacy, patient reported outcomes, disease severity, safety, and quality of evidence (relative weight >0.075 each). Compared to all participants, policymakers gave more weight to budget impact and quality of evidence. The quantitative appraisal of a pulmonary heart sensor revealed differences in scoring performance of this intervention at the criteria level between stakeholder groups. The highest value estimate of the sensor reached 0.68 (on a scale of 0 to 1, 1 representing maximum value) for industry representatives and the lowest value of 0.40 was reported for policymakers, compared to 0.48 for all participants. Participants indicated that most qualitative criteria should be considered and their impact on the quantitative appraisal was captured transparently. Conclusions The study identified important variations in perspectives across German stakeholders when appraising a healthcare intervention and revealed that MCDA can demonstrate the value of a specified technology for all participating stakeholders. Better understanding of these differences at the criteria level, in particular between policymakers and industry representatives, is important to focus innovation aligned with patient health and healthcare system values and constraints