Guidance on the integrated assessment of complex health technologies: the INTEGRATE-HTA model

Challenges in assessments of health technologies In recent years there have been major advances in the development of health technology assessment (HTA). However, HTA still has certain limitations when assessing technologies which are complex, i.e. consist of several interacting components, target different groups or organizational levels, have multiple and variable outcomes, and/or permit a certain degree of flexibility or tailoring (Craig et al., 2008), fi are context-dependent - current HTA usually focusses on the technology, not on the system within which it is used, fi perform differently depending on the way they are implemented, fi have different effects on different individuals. Furthermore, HTA usually assesses and appraises aspects side-by-side, while decision-making needs an integrated perspective on the value of a technology. In the EU-funded INTEGRATE-HTA project, we developed concepts and methods to deal with these challenges, which are described in six guidances. Because of the interactions, an integrated assessment needs to start from the beginning of the assessment. This guidance provides a systematic five-step-process for an integrated assessment of complex technologies (the INTEGRATE-HTA Model). Purpose and scope of the guidance The aim of the INTEGRATE-HTA project is to provide concepts and methods that enable a patient-centred, comprehensive, and integrated assessment of complex health technologies. The purpose of this guidance is to structure the overall HTA-process. The INTEGRATE-HTA Model outlines an integrated scoping process, a coordinated application of assessment methods for different aspects and an integrated and structured decision-making process. It is intended for HTA agencies, HTA researchers and those engaged in the evaluation of complex health technologies. As it links the assessment to the decision-making process, it also addresses HTA commissioners and other stakeholders using or planning HTAs. While all technologies are arguably complex, some are more complex than others. Applying this guidance might lead to a more thorough and therefore more time-consuming process. Depending on the degree of complexity, one might choose to follow the whole process as described in this guidance, or only focus on certain steps. The guidance provides an operational definition to assess the complexity of technologies which can be used to identify specific aspects that will need more attention than others. What the guidance does not provide is a post-hoc solution for assessments that have already been completed. | 6 Development of the guidance The INTEGRATE-HTA Model presented in this guidance was developed based on a systematic literature search on approaches for integration, on the experiences of traditional HTAs, as well as on the other methodological guidances developed in the INTEGRATE-HTA project. It was tested in a case study on palliative care and iteratively revised during the practical application. The guidance was again revised after internal and external peer-review. Application of this guidance For a comprehensive integrated assessment of a complex technology, we developed a five-step process, the INTEGRATE-HTA model. In Step 1, the HTA objective and the technology are defined with the support from a panel of stakeholders. An initial logic model is developed in Step 2. The initial logic model provides a structured overview of the technology, the context, implementation issues, and relevant patient groups. It then frames the assessment of the effectiveness, as well as economic, ethical, legal, and socio-cultural aspects in Step 3. In Step 4, a graphical overview of the assessment results, structured by the logic model, is provided. Step 5 is a structured decision-making process informed by the HTA (and is thus not formally part of the HTA, but follows it). fi Step 1: In step 1, the technology under assessment and the objective of the HTA are defined. Especially for complex technologies, such as palliative care, the definition of the technology alone is a challenge that must not be underestimated. It is recommended to do this based on a tentative literature review and with the support of stakeholder advisory panels (SAPs) which should comprise clinical experts, academics, patients, possibly their relatives and/or other caretakers, and the public. The setting of an objective considering all relevant aspects of complexity and structured by assessment criteria is important. The assessment criteria will usually reflect values of the stakeholders as well as the input from the theoretical, methodological and empirical literature. fi Step 2: In step 2, an initial logic model is developed (see Guidance on the use of logic models in health technology assessments of complex interventions). The model provides a structured overview on participants, interventions, comparators, and outcomes. Parallel to this, groups of patients that are distinguished by different preferences and treatment moderators (see Guidance for the assessment of treatment moderation and patients’ preferences) are identified. Specific context and implementation issues are also identified as part of the initial logic model (see Guidance for the Assessment of Context and Implementation in Health Technology Assessments (HTA) and Systematic Reviews of Complex Interventions). The product of this step is the logic model as a graphical representation of all aspects and their interactions that are relevant for the assessment of the complex technology. fi Step 3: In step 3, the logic model serves as a conceptual framework that guides the evidence assessment. Depending on the specific aspect (e.g. effectiveness, economic, ethical, socio-cultural, or legal aspects) different methods are available for the assessment (see Guidance for assessing effectiveness, economic aspects, ethical aspects, socio-cultural aspects and legal aspects in complex technologies). The outputs of step 3 are evidence reports and standardized evidence summaries for each assessment aspect (e.g. report on economics, report on ethical aspects, etc.). fi Step 4: In step 4, the assessment results of step 3 are structured using the logic model developed in step 2. Whereas the initial logic model in step 2 specifies what evidence is relevant, the extended logic model to assist decision-making in step 4 visualizes the assessment results as well as the interaction with respect to the HTA objectives. It also allows for the consideration of different scenarios depending on the variation in context, implementation and patient characteristics. 7 | fi Step 5: Step 5 involves a structured decision-making process and is not an integral part of the HTA in the narrow sense. Decision-making can be supported by applying quantitative e.g. MCDA- (Multi-criteria decision analysis) or qualitative decision support tools. Flexibility in the application of these tools by the decision committee is crucial, taking different decision settings and evidence needs into consideration. Conclusions In current HTA, different aspects are usually assessed and presented independent of each other. Context, implementation issues and patient characteristics are rarely considered. The INTEGRATE-HTA Model enables a coordinated assessment of all these aspects and addresses their interdependencies. The perspective of stakeholders such as patients and professionals with their values and preferences is integrated in the INTEGRATE-HTA Model to obtain HTA results that are meaningful for all relevant stakeholders. Finally, health policy makers obtain an integrated perspective of the assessment results to achieve fair and legitimate conclusions at the end of the HTA process. The application of the model will usually require more time and resources than traditional HTA. An initial assessment of the degree and the character of complexity of a technology might be helpful to decide whether or not the whole process or only specific elements will be applied

MAPKinase inhibition after failure of immune checkpoint blockade in patients with advanced melanoma – an evaluation of the multicenter prospective skin cancer registry ADOREG

Objectives: Forty to sixty percent of patients with advanced melanoma show primary resistance to PD-1-based immunotherapy, 30-40% of initial responders also progress. Here, we evaluated the outcome of second-line targeted therapy (TT) after progression on PD-1-based immune checkpoint inhibition (ICI) in BRAFV600-mutated melanoma. In addition, we report data on the activity of re-exposure with PD-1-based regimes. Methods: Patients with advanced (non- resectable stage III or IV, AJCC 2017, 8th edition) melanoma progressing on PD-1-based ICI (nivolumab, pembrolizumab or ipilimumab plus nivolumab) and receiving second-line BRAF plus MEK inhibition were identified from the prospective multicenter skin cancer registry ADOREG. Results: We identified 108 patients with unresectable stage III or stage IV melanoma progressing on first-line ICI (nivolumab, pembrolizumab or ipilimumab plus nivolumab) and receiving second-line combined BRAF/MEK inhibition. Seventy- three percent of the cohort presented with primary PD-1 resistant disease. Median progression-free survival ( PFS) on ICI was 2.6 (95% CI 2.2-2.9) months. Median PFS on subsequent TT was 6.6 (95% CI 5.4 -7.8) months. Median OS from start of second-line TT was 16.0 (95% CI 11.2-20.8) months. The 3-year PFS and OS rates on second-line TT were 16% and 30%. The objective response rate (ORR) and disease control rate (DCR) to TT were 42.6% and 55.6%. In patients with brain metastases, the ORR and DCR were 31.4% and 43.1%. Patients without brain metastases showed an ORR and DCR of 52.6% and 66.7%, respectively. Response to first-line ICI was associated with a numerically higher ORR and DCR to second-line TT and improved OS on TT. Twenty-three patients received third-line ICI of whom two patients showed an objective response. Conclusions: BRAF plus MEK inhibition shows meaningful activity and outcome in patients with advanced melanoma resistant to anti-PD-1- based immunotherapy. Rates of long- term benefit and survival in our study were similar to those reported for treatment-naive patients receiving first-line MAPKi

Brain metastasis and survival outcomes after first-line therapy in metastatic melanoma: a multicenter DeCOG study on 1704 patients from the prospective skin cancer registry ADOREG

Background Despite the availability of effective systemic therapies, a significant number of advanced melanoma patients develops brain metastases. This study investigated differences in incidence and time to diagnosis of brain metastasis and survival outcomes dependent on the type of first-line therapy.Methods Patients with metastatic, non-resectable melanoma (AJCCv8 stage IIIC–V) without brain metastasis at start of first-line therapy (1L-therapy) were identified from the prospective multicenter real-world skin cancer registry ADOREG. Study endpoints were incidence of brain metastasis, brain metastasis-free survival (BMFS), progression-free survival (PFS), and overall survival (OS).Results Of 1704 patients, 916 were BRAF wild-type (BRAFwt) and 788 were BRAF V600 mutant (BRAFmut). Median follow-up time after start of 1L-therapy was 40.4 months. BRAFwt patients received 1L-therapy with immune checkpoint inhibitors (ICI) against CTLA-4+PD-1 (n=281) or PD-1 (n=544). In BRAFmut patients, 1L-therapy was ICI in 415 patients (CTLA-4+PD-1, n=108; PD-1, n=264), and BRAF+MEK targeted therapy (TT) in 373 patients. After 24 months, 1L-therapy with BRAF+MEK resulted in a higher incidence of brain metastasis compared with PD-1±CTLA-4 (BRAF+MEK, 30.3%; CTLA-4+PD-1, 22.2%; PD-1, 14.0%). In multivariate analysis, BRAFmut patients developed brain metastases earlier on 1L-therapy with BRAF+MEK than with PD-1±CTLA-4 (CTLA-4+PD-1: HR 0.560, 95% CI 0.332 to 0.945, p=0.030; PD-1: HR 0.575, 95% CI 0.372 to 0.888, p=0.013). Type of 1L-therapy, tumor stage, and age were independent prognostic factors for BMFS in BRAFmut patients. In BRAFwt patients, tumor stage was independently associated with longer BMFS; ECOG Performance status (ECOG-PS), lactate dehydrogenase (LDH), and tumor stage with OS. CTLA-4+PD-1 did not result in better BMFS, PFS, or OS than PD-1 in BRAFwt patients. For BRAFmut patients, multivariate Cox regression revealed ECOG-PS, type of 1L-therapy, tumor stage, and LDH as independent prognostic factors for PFS and OS. 1L-therapy with CTLA-4+PD-1 led to longer OS than PD-1 (HR 1.97, 95% CI 1.122 to 3.455, p=0.018) or BRAF+MEK (HR 2.41, 95% CI 1.432 to 4.054, p=0.001), without PD-1 being superior to BRAF+MEK.Conclusions In BRAFmut patients 1L-therapy with PD-1±CTLA-4 ICI resulted in a delayed and less frequent development of brain metastasis compared with BRAF+MEK TT. 1L-therapy with CTLA-4+PD-1 showed superior OS compared with PD-1 and BRAF+MEK. In BRAFwt patients, no differences in brain metastasis and survival outcomes were detected for CTLA-4+PD-1 compared with PD-1

Motive, Methoden und Erfahrungen der aktiven Beteiligung von Patient*innen an klinischer Forschung

Hintergrund: Zunehmend wird gefordert, Patient*innen an der Planung, Durchführung und Dissemination von Forschung zu beteiligen. Der Umsetzung von PPI stehen jedoch Herausforderungen entgegen. Ziel der Dissertation war es, zum Wissen, wie PPI gelingen kann, beizutragen. Fokussiert wurde auf die Perspektiven der Beteiligten sowie Methoden zur Beteiligung einer „oft übersehenen“ Zielgruppe, hier die der älteren Patient*innen. Methode: In zwei qualitativen Studien wurde untersucht, (I) was Patient*innen und Forschende motiviert, (a) PPI umzusetzen und (b) welche Erwartungen sie haben, sowie (II) welche Erfahrungen sie mit der Beteiligung machen. In einem Review wurden systematisch (III) Methoden und fördernde Strategien zur Beteiligung älterer Patient*innen identifiziert und analysiert. Ergebnisse: (I) Die Motive und Erwartungen der beteiligten Patient*innen und Forschenden waren in einigen Punkten identisch, unterschieden sich jedoch u.a. hinsichtlich des Verständnisses ihrer Rollen. (II) Auch die Erfahrungen der Beteiligten unterschieden sich in einigen Aspekten. Während beide Gruppen die Atmosphäre und Zusammenarbeit als angenehm empfanden und Trainingsbedarfe feststellten, wurden die Rollen, die Intensität und der Nutzen der Beteiligung unterschiedlich wahrgenommen. (III) PPI mit älteren Patient*innen ist mit verschiedenen Methoden wie Referenzgruppen, Workshops oder schriftlichen Erhebungen, umsetzbar. Sie geht jedoch mit speziellen Herausforderungen einher. Fazit: Zum Gelingen von PPI kann beitragen, dass die beteiligten Stakeholder*innen 1) gute Beziehungen aufbauen, sich über ihre Motive und Erwartungen austauschen und die Ziele und Arbeitsweisen der Beteiligung vereinbaren, 2) Trainingsbedarfe berücksichtigen, 3) die Wahl von Methoden, Zeitpunkt und Teilnehmenden in Hinblick auf die Ziele sowie die Vor- und Nachteile verschiedener Ansätze reflektieren, Patient*innen früh beteiligen und Diversität in der Auswahl der beteiligten Perspektiven anstreben, 4) PPI rechtzeitig planen und in den regulären Studienprozess einbinden, 5) Flexibilität ermöglichen und 6) ausreichend Ressourcen zur Verfügung haben

Methods for Involving Older People in Health Research—A Review of the Literature

Demographic change has increased the need for research on healthcare for older people. Recently there has been a growing awareness that research might benefit from actively involving patients and the public in study design and conduct. Besides empowering patients and democratizing research, involvement enhances the quality of research and the development of equitable healthcare solutions. Little is known about how to involve older people. This review aims to support scientists intending to involve older people in health research by systematically identifying and describing studies involving older people and analyzing associated facilitators and challenges. Old people were operationalized as people living with old-age-related conditions. We conducted a systematic search in PubMed, CINAHL (Cumulative Index to Nursing and Allied Health Literature), and Cochrane library for the period 2007 to July 2017 and also manually searched reference lists of the nine retrieved articles and other relevant sources. While involvement of older people in research is feasible, specific challenges related to this group need be taken into account. Strategies to enhance effective involvement comprise a thoughtful choice of location, use of visualization and accessible communication, building good relationships and flexible approaches. Further research is needed on the involvement of people in care homes or with vision, hearing or mobility limitations

Patient involvement in clinical trials: motivation and expectations differ between patients and researchers involved in a trial on urinary tract infections

Plain English summary Patients should be involved in the design, conduct and dissemination of research that affects them. Patient involvement leads to empowerment and enhances the quality of research. Differing motives and expectations between researchers and patients involved can hamper involvement. We wanted to learn more about patients’ and researchers’ motives and expectations in order to improve the benefits of involvement for all parties. We implemented a patient board with ten patients and five researchers for a trial on urinary tract infections (UTIs). We asked each patient and researcher about his or her motivation and expectations regarding the patient board. We found that patients’ motivations included the wish to improve the treatment of UTIs, to support patient involvement as a principle, and to enhance the benefit of others. Furthermore they were interested in learning how a patients’ board works and in exchanging with peers and scientists. In addition, a (modest) monetary incentive for involvement was welcomed.Researchers were motivated by the possibility to improve research and to contribute to the empowerment of patients. They also wanted to enhance their career opportunities, to learn more about patient involvement and to meet the increasing demand for it. Some patients expressed insecurity about their roles and tasks in the patient board. Among the researchers, some envisaged a rather passive role for themselves in the patient board while others expected to take over a more active role. Researchers emphasized that the ways and the means of communication between the researchers and the patients should be explicitly discussed. Abstract Background It has been increasingly recognized that patients should be actively involved in the design, conduct and dissemination of research. Besides empowering patients and democratizing research, involvement can enhance the quality of research and the development of equitable healthcare solutions. Differing motives and expectations between researchers and involved patients can hamper the conduct of involvement. However, little is known about patients’ and researchers’ motivations for involvement. Our aim was to study the motivation and expectations of patients and researchers towards patient and public involvement (PPI). Methods We implemented a patient board comprising ten patients and five researchers for a randomized controlled trial on the treatment of urinary tract infections (UTI). Prior to the first board meeting, we conducted telephone interviews with all researchers and patients regarding their motivation for involvement in the patient board and their expectations. The interviews were analyzed using thematic qualitative text analysis. Results Patients’ motivations included interest in improving UTI treatment, in supporting PPI, engaging for the benefit of others, exchanging with peers and scientists as well as in the methods of the board and the monetary incentive. Researchers wanted to improve research, enhance their professional development, empower patients, meet the formal demand for PPI, and learn about PPI. Regarding expectations, patients expressed insecurities about their roles, tasks and topics of discussion. They wished for an open exchange and hoped their involvement would make an impact. Researchers’ expectations for their own roles ranged between being a rather passive supporting force and active engagement in the board. The question of how to ensure the communication between the trial team and the patient board was of high importance for the researchers. Conclusions Patients’ and researchers’ motives and expectations were similar in some aspects but differed regarding agenda setting and understanding of their roles. Getting to know patients’ and researchers’ motivations and expectations at the beginning allowed us to anticipate potential conflicts or disappointments early on and to take them into consideration during the conduct of our PPI

Problemorientiertes Lernen (PoL)

Strulik T. Problemorientiertes Lernen (PoL). In: Gerhardus A, Kolip P, Schilling I, Munko T, Schlingmann K, eds. Lehren und Lernen in den Gesundheitswissenschaften. Ein Praxishandbuch. Bern: Hogrefe; 2019: 104-109

E-Learning und Blended Learning in Groẞveranstaltungen. Ein Praxisbericht

Buschmeier M. E-Learning und Blended Learning in Groẞveranstaltungen. Ein Praxisbericht. In: Gerhardus A, Kolip P, Schilling I, Munko T, Schlingmann K, eds. Lehren und Lernen in den Gesundheitswissenschaften. Ein Praxishandbuch. Bern : Hogrefe; 2019: 126-135