Problems and Promises of Health Technologies: The Role of Early Health Economic Modeling

Background: To assess whether early health economic modeling helps to distinguish those healthcare innovations that are potentially cost-effective from those that are not potentially cost-effective. We will also study what information is retrieved from the health economic models to inform further development, research and implementation decisions.Methods: We performed secondary analyses on an existing database of 32 health economic modeling assessments of 30 innovations, performed by our group. First, we explored whether the assessments could distinguish innovations with potential cost-effectiveness from innovations without potential cost-effectiveness. Second, we explored which recommendations were made regarding development, implementation and further research of the innovation. Results: Of the 30 innovations, 1 (3%) was an idea that was not yet being developed and 14 (47%) were under development. Eight (27%) innovations had finished development, and another 7 (23%) innovations were on the market. Although all assessments showed that the innovation had the potential to become cost-effective, due to improved patient outcomes, cost savings or both, differences were found in the magnitude of the potential benefits, and the likelihood of reaching this potential. The assessments informed how the innovation could be further developed or positioned to maximize its cost-effectiveness, and informed further research.Conclusion: The early health economic assessments provided insight in the potential cost-effectiveness of an innovation in its intended context, and the associated uncertainty. None of the assessments resulted in a firm ‘no-go’ recommendation, but recommendations could be provided on further research and development in order to maximize value for money

Guidance on the use of logic models in health technology assessments of complex interventions

Challenges in assessments of health technologies In recent years there have been major advances in the development of health technology assessment (HTA). However, HTA still has certain limitations when assessing technologies, which fi are complex, i.e. consist of several interacting components, target different groups or organisational levels, have multiple and variable outcomes, and/or permit a certain degree of flexibility or tailoring; fi are context-dependent, with HTA usually focusing on the technology rather than on the system within which it is used; fi perform differently depending on the way they are implemented; and/or fi have distinct effects on different individuals. Logic models are one important means of conceptualising and handling complexity in HTAs or systematic reviews (SRs) of complex technologies, as well as integrating the findings of multi-component HTAs. A logic model is described as “… a graphic description of a system … designed to identify important elements and relationships within that system”. When evaluating complex health technologies, logic models can serve an instrumental purpose at every stage of the HTA/SR process, from scoping the topic of the HTA/SR, including formulating the question and defining the intervention; conducting the HTA/SR; interpreting results and making the HTA/SR relevant for decision makers to implement in policy and practice. Purpose and scope of the guidance This guidance is targeted at commissioners, producers and users of guidelines, HTAs and SRs with an interest in using logic models as an overarching framework for their work. It aims to make the use of logic models as straightforward as possible by facilitating the systematic identification or development as well as utilisation of different types and sub-types of logic models. In principle, logic models are a useful tool in any kind of SR or HTA, as they aid with the conceptualisation of the intervention and the review question. This is particularly useful for complex technologies, where conceptualising the intervention and its implementation within a system is critical. In addition, logic models can enhance communication within the HTA/SR team and with relevant stakeholders. Three types of logic model are described: With a priori logic models the logic model is specified upfront and remains unchanged during the HTA/SR process. With iterative logic models the logic model is subject to continual modification throughout the course of an HTA/SR. The staged logic model harnesses the strengths of both a priori and iterative approaches by pre-specifying revision points at which major data inputs are anticipated. In | 6 addition, two subtypes are identified, namely logic models that seek to represent structure (system-based) and those that focus on processes or activities (process-orientated). This guidance offers direction on how to choose between distinct types and sub-types of logic models, describes each logic model type and its application in detail, and provides templates for getting started with the development of an HTA/SR-specific logic model. Development of the guidance This guidance was informed by a combination of (i) systematic searches for published examples of logic models supplemented by purposive sampling of iterative logic modelling approaches; (ii) searches for existing guidance on the use of logic models in primary research, SRs and HTAs; (iii) development of two draft templates for system-based and process-orientated logic models in an iterative process within the research team and in consultation with external methodological experts; and (iv) application of these draft templates in multiple SRs and one HTA of different complex health technologies covering technical, educational and policy interventions in environmental health, e-learning for health professionals and models of palliative care. Application of this guidance For a comprehensive integrated assessment of a complex technology we have developed a five step process, the INTEGRATE-HTA model. In Step 1 the HTA objective and the technology are defined with the support from a panel of stakeholders. A system-based logic model is developed in Step 2. It provides a structured overview of technology, the context, implementation issues, and relevant patient groups. It then frames the assessment of the effectiveness, as well as economic, ethical, legal, and socio-cultural aspects in Step 3. In Step 4 a graphical overview of the assessment results, structured by the logic model, is provided. Step 5 is a structured decision-making process informed by the HTA (and is thus not formally part of the HTA but follows it). Logic models therefore form an integral element of the INTEGRATE-HTA model but may also be useful in individual steps. This guidance starts off by offering support in identifying and, as needed, adapting existing logic models from the literature or developing an HTA-/SR-specific logic model de novo. In either case, the user will need to decide upfront whether to pursue an a priori, staged or iterative approach to logic modelling, and the guidance offers criteria on how to decide between these distinct types of logic modelling. The system-based and process-orientated logic model templates provide a starting point for the de novo development of either type of logic model. The guidance also discusses the advantages and drawbacks of adopting the system-based or process-orientated sub-type, and offers some general considerations in applying logic models, such as the variety of data sources used, transparency in reporting and necessary trade-offs between comprehensiveness and complexity of the logic model in communicating with stakeholders. For a priori logic modelling, a six-step process comprises: (1) defining the PICO elements of the HTA/SR as well as relevant aspect of context and implementation; (2) deciding on a system- vs. process-orientated logic model subtype with the former focusing on a conceptualization of the question and the latter more concerned with an explanation of the pathways from the intervention to the outcomes; (3) populating the logic model template with information obtained through literature searches, discussions within the author team and consultations with content experts; (4) asking stakeholders for input and refining the logic model accordingly; (5) repeating steps 3 and 4 until all members of the author team agree that the logic model accurately represents the framework for the specific HTA/SR; and (6) publishing the final logic model with the protocol of the HTA or SR. This logic model remains unchanged during the HTA/SR process. For iterative logic modelling, a five-step process includes: (1) creating an initial logic model as a starting point for subsequent exploration, where a logic model template is used to create an initial logic model de novo; (2) identifying data on the whole system or entire process, or on individual components of the model, where data may come from stakeholders, the review team, ongoing primary research or the published literature; (3) making 7 | changes to the initial logic model repeatedly and at any point of the review and documenting these changes carefully; (4) creating a new numbered version of the logic model, where changes are considered substantive or stepwise; and (5) recording a definitive version of the logic model for the purpose of publication within the final HTA/SR report. It is recognised that this version of the logic model is only definitive with regard to the specific project timeframe and may well be subject to subsequent modification by the HTA/SR team, or indeed by other teams. For staged logic modelling, a four-step process consists of: (1) developing an initial logic model, using one of the templates and various mechanisms to populate them, in particular input from stakeholders and literature searches; (2) pre-specifying points within the HTA/SR process at which significant inputs, defined in terms of quantity or importance, are likely to have an impact on the structure and content of the HTA/SR and thus the logic model; (3) revisiting the logic model at the pre-specified review and revision points, and creating new and clearly labelled versions, documenting how and based on which data sources changes were made; and (4) presenting selected versions of the logic model, as a minimum the initial and the final logic models, in the HTA/ SR report. Conclusions Logic models are an important tool when conducting HTAs or SRs of complex health technologies, as they enhance transparency on underlying assumptions and help understand complexity by depicting the entire system, its parts and the interactions between intervention and outcomes; they also play a key role in integrating across different parts of a multi-component HTA. Nonetheless, logic models are not a panacea in addressing or resolving complexity and each type shows its specific strengths and limitations. This guidance provides a stateof-the-art overview of current practices in the use of logic models within HTAs and SRs. By providing templates for generating a logic model de novo, it aims to make the process of logic model development and application as straightforward as possible. Certain types and sub-types of logic models are more or less suitable depending on the technology concerned and the HTA/SR question addressed and approach pursued. This guidance offers a series of considerations on how to choose between a priori, iterative and staged logic models, illustrated with example applications of each type

Why is it so difficult to integrate ethics in Health Technology Assessment (HTA)? The epistemological viewpoint

Ethics has been identified as a key element in Health Technology Assessment (HTA) since its conception. However, ethical issues are still not frequently addressed explicitly in HTA. Several valuable reasons have been identified. The basis of the article is the claim that ethics is often not part of HTA for “epistemological reasons”. Hence, the main aim of the contribution is to explore in more details and emphasize them by using the fact/value dichotomy. Our conclusion is that current HTA configuration is dominantly based on the comparison among objective and empirically testable “facts”, whilst ethics is not empirically testable. In this sense, there is a sort of “epistemological gap”, which can explain why it is so difficult to integrate ethics in HTA. We suggest that the epistemological differences among the various domains of HTA are addressed more explicitly

Stakeholder involvement throughout health technology assessment: an example from palliative care

Objectives: Internationally, funders require stakeholder involvement throughout health technology assessment (HTA). We report successes, challenges, and lessons learned from extensive stakeholder involvement throughout a palliative care case study that demonstrates new concepts and methods for HTA. Methods: A 5-step “INTEGRATE-HTA Model” developed within the INTEGRATE-HTA project guided the case study. Using convenience or purposive sampling or directly / indirectly identifying and approaching individuals / groups, stakeholders participated in qualitative research or consultation meetings. During scoping, 132 stakeholders, aged ≥ 18 years in seven countries (England, Italy, Germany, The Netherlands, Norway, Lithuania, and Poland), highlighted key issues in palliative care that assisted identification of the intervention and comparator. Subsequently stakeholders in four countries participated in face–face, telephone and / or video Skype meetings to inform evidence collection and / or review assessment results. An applicability assessment to identify contextual and implementation barriers and enablers for the case study findings involved twelve professionals in the three countries. Finally, thirteen stakeholders participated in a mock decision-making meeting in England. Results: Views about the best methods of stakeholder involvement vary internationally. Stakeholders make valuable contributions in all stages of HTA; assisting decision making about interventions, comparators, research questions; providing evidence and insights into findings, gap analyses and applicability assessments. Key challenges exist regarding inclusivity, time, and resource use. Conclusion: Stakeholder involvement is feasible and worthwhile throughout HTA, sometimes providing unique insights. Various methods can be used to include stakeholders, although challenges exist. Recognition of stakeholder expertise and further guidance about stakeholder consultation methods is needed

The development of CHAMP : a checklist for the appraisal of moderators and predictors

BACKGROUND: Personalized healthcare relies on the identification of factors explaining why individuals respond differently to the same intervention. Analyses identifying such factors, so called predictors and moderators, have their own set of assumptions and limitations which, when violated, can result in misleading claims, and incorrect actions. The aim of this study was to develop a checklist for critically appraising the results of predictor and moderator analyses by combining recommendations from published guidelines and experts in the field. METHODS: Candidate criteria for the checklist were retrieved through systematic searches of the literature. These criteria were evaluated for appropriateness using a Delphi procedure. Two Delphi rounds yielded a pilot checklist, which was tested on a set of papers included in a systematic review on reinforced home-based palliative care. The results of the pilot informed a third Delphi round, which served to finalize the checklist. RESULTS: Forty-nine appraisal criteria were identified in the literature. Feedback was obtained from fourteen experts from (bio)statistics, epidemiology and other associated fields elicited via three Delphi rounds. Additional feedback from other researchers was collected in a pilot test. The final version of our checklist included seventeen criteria, covering the design (e.g. a priori plausibility), analysis (e.g. use of interaction tests) and results (e.g. complete reporting) of moderator and predictor analysis, together with the transferability of the results (e.g. clinical importance). There are criteria both for individual papers and for bodies of evidence. CONCLUSIONS: The proposed checklist can be used for critical appraisal of reported moderator and predictor effects, as assessed in randomized or non-randomized studies using individual participant or aggregate data. This checklist is accompanied by a user's guide to facilitate implementation. Its future use across a wide variety of research domains and study types will provide insights about its usability and feasibilit

Lay and professional stakeholder involvement in scoping palliative care issues: Methods used in seven European countries

BACKGROUND: Stakeholders are people with an interest in a topic. Internationally, stakeholder involvement in palliative care research and health technology assessment requires development. Stakeholder involvement adds value throughout research (from prioritising topics to disseminating findings). Philosophies and understandings about the best ways to involve stakeholders in research differ internationally. Stakeholder involvement took place in seven countries (England, Germany, Italy, Lithuania, the Netherlands, Norway and Poland). Findings informed a project that developed concepts and methods for health technology assessment and applied these to evaluate models of palliative care service delivery. AIMS: To report on stakeholder involvement in the INTEGRATE-HTA project and how issues identified informed project development. DESIGN: Using stakeholder consultation or a qualitative research design, as appropriate locally, stakeholders in seven countries acted as 'advisors' to aid researchers' decision making. Thematic analysis was used to identify key issues across countries. SETTING/PARTICIPANTS: A total of 132 stakeholders (82 professionals and 50 'lay' people) aged ⩾18 participated in individual face-to-face or telephone interviews, consultation meetings or focus groups. RESULTS: Different stakeholder involvement methods were used successfully to identify key issues in palliative care. A total of 23 issues common to three or more countries informed decisions about the intervention and comparator of interest, sub questions and specific assessments within the health technology assessment. CONCLUSION: Stakeholders, including patients and families undergoing palliative care, can inform project decision making using various involvement methods according to the local context. Researchers should consider local understandings about stakeholder involvement as views of appropriate and feasible methods vary. Methods for stakeholder involvement, especially consultation, need further development

ESUR/ESUI consensus statements on multi-parametric MRI for the detection of clinically significant prostate cancer: quality requirements for image acquisition, interpretation and radiologists’ training

Objectives: This study aims to define consensus-based criteria for acquiring and reporting prostate MRI and establishing prerequisites for image quality. Methods: A total of 44 leading urologists and urogenital radiologists who are experts in prostate cancer imaging from the European Society of Urogenital Radiology (ESUR) and EAU Section of Urologic Imaging (ESUI) participated in a Delphi consensus process. Panellists completed two rounds of questionnaires with 55 items under three headings: image quality assessment, interpretation and reporting, and radiologists’ experience plus training centres. Of 55 questions, 31 were rated for agreement on a 9-point scale, and 24 were multiple-choice or open. For agreement items, there was consensus agreement with an agreement ≥ 70% (score 7–9) and disa

Integrated assessment of home based palliative care with and without reinforced caregiver support: ‘A demonstration of INTEGRATE-HTA methodological guidances’

About this Executive Summary This comprehensive executive summary reports on a case study designed to demonstrate the application of a number of the key concepts and methods developed in the INTEGRATE-HTA project to the assessment of complex technologies. The case study focuses on models of home based palliative care with and without an additional element of caregiver support, known as reinforced and non-reinforced home based palliative care respectively. What is already known about the topic With changing disease patterns in Europe, increasingly complex health care technologies, such as palliative care, have gained importance. Current HTA methods rarely take account of wider legal, ethical and socio-cultural issues or context and implementation and are not adequately equipped to assess highly complex technologies, despite considerable progress in recent years. As a result, HTA is rarely applied to highly complex health technologies. What this case study report adds The INTEGRATE-HTA project developed concepts and methods for the assessment of complex technologies, taking into account legal, ethical and socio-cultural issues as well as context and implementation. This case study is designed to demonstrate the application of a number of the key concepts and methods developed in the INTEGRATE-HTA project to home based palliative care, with and without an additional element of caregiver support, as an example of a complex technology; known as “reinforced” and “non-reinforced” home based palliative care respectively. The INTEGRATE-HTA model, developed to enable integration of relevant assessment aspects, is used to structure this report. The case study reports on the application of some of the concepts and methods developed within the INTEGRATE-HTA project to the assessment of effectiveness as well as economic, sociocultural, ethical, and legal aspects; patient preferences and patient-specific moderators of treatment and context and implementation issues related to reinforced and non-reinforced home based palliative care. The highly complex nature of reinforced and non-reinforced home based palliative care is illustrated through an assessment of complexity characteristics. | 4 The case study also involved extensive lay and professional stakeholder involvement, using a variety of methods, to inform the HTA process at key stages throughout the project. Key messages are highlighted for both HTA research and for palliative care. The implications for research and practice The concepts and methods developed in the INTEGRATE-HTA project have been shown to be feasible and to have the potential to offer added value, but require further development and application in the assessment of other complex technologies. Reinforced and non-reinforced models palliative care are highly complex. The case study findings offer some insights into their effectiveness as well as economic, sociocultural, ethical, and legal issues; patient preferences and patient-specific moderators of treatment as well as context and implementation issues

ESUR/ESUI consensus statements on multi-parametric MRI for the detection of clinically significant prostate cancer: quality requirements for image acquisition, interpretation and radiologists’ training

Funder: Radboud University Medical CenterAbstract: Objectives: This study aims to define consensus-based criteria for acquiring and reporting prostate MRI and establishing prerequisites for image quality. Methods: A total of 44 leading urologists and urogenital radiologists who are experts in prostate cancer imaging from the European Society of Urogenital Radiology (ESUR) and EAU Section of Urologic Imaging (ESUI) participated in a Delphi consensus process. Panellists completed two rounds of questionnaires with 55 items under three headings: image quality assessment, interpretation and reporting, and radiologists’ experience plus training centres. Of 55 questions, 31 were rated for agreement on a 9-point scale, and 24 were multiple-choice or open. For agreement items, there was consensus agreement with an agreement ≥ 70% (score 7–9) and disagreement of ≤ 15% of the panellists. For the other questions, a consensus was considered with ≥ 50% of votes. Results: Twenty-four out of 31 of agreement items and 11/16 of other questions reached consensus. Agreement statements were (1) reporting of image quality should be performed and implemented into clinical practice; (2) for interpretation performance, radiologists should use self-performance tests with histopathology feedback, compare their interpretation with expert-reading and use external performance assessments; and (3) radiologists must attend theoretical and hands-on courses before interpreting prostate MRI. Limitations are that the results are expert opinions and not based on systematic reviews or meta-analyses. There was no consensus on outcomes statements of prostate MRI assessment as quality marker. Conclusions: An ESUR and ESUI expert panel showed high agreement (74%) on issues improving prostate MRI quality. Checking and reporting of image quality are mandatory. Prostate radiologists should attend theoretical and hands-on courses, followed by supervised education, and must perform regular performance assessments. Key Points: • Multi-parametric MRI in the diagnostic pathway of prostate cancer has a well-established upfront role in the recently updated European Association of Urology guideline and American Urological Association recommendations. • Suboptimal image acquisition and reporting at an individual level will result in clinicians losing confidence in the technique and returning to the (non-MRI) systematic biopsy pathway. Therefore, it is crucial to establish quality criteria for the acquisition and reporting of mpMRI. • To ensure high-quality prostate MRI, experts consider checking and reporting of image quality mandatory. Prostate radiologists must attend theoretical and hands-on courses, followed by supervised education, and must perform regular self- and external performance assessments