Evaluating the Validation Process:Embracing Complexity and Transparency in Health Economic Modelling

Reimbursement decisions and price negotiation of healthcare interventions often rely on health economic model results. Such decisions affect resource allocation, patient outcomes and future healthcare choices. To ensure optimal decisions, assessing the validity of health economic models may be crucial. Validation involves much more than identifying (and hopefully correcting) errors in the model implementation. It also includes assessing the conceptual validity of the model and validation of the model input data, and checking whether the model’s predictions align sufficiently well with real-world data. In the context of health economics, validation can be defined as “the act of evaluating whether a model is a proper and sufficient representation of the system it is intended to represent in view of an application”, meaning that the model complies with what is known about the system and its outcomes provide a robust basis for decision making.[...]Validation of health economic models should be seen as a critical component of evidence-based decision making in healthcare. However, as of today, it still faces several important challenges, including the lack of consensus guidance and standardised procedures, the need for greater rigour or the question of who should oversee the validation process. To address these challenges, we encourage model developers, agencies requiring models for their decision making and editors of journals that publish models to recommend the use of state-of-the-art tools for reporting (and conducting) validations of health economic models, such as those mentioned in this editorial

Evaluating the Validation Process:Embracing Complexity and Transparency in Health Economic Modelling

Reimbursement decisions and price negotiation of healthcare interventions often rely on health economic model results. Such decisions affect resource allocation, patient outcomes and future healthcare choices. To ensure optimal decisions, assessing the validity of health economic models may be crucial. Validation involves much more than identifying (and hopefully correcting) errors in the model implementation. It also includes assessing the conceptual validity of the model and validation of the model input data, and checking whether the model’s predictions align sufficiently well with real-world data. In the context of health economics, validation can be defined as “the act of evaluating whether a model is a proper and sufficient representation of the system it is intended to represent in view of an application”, meaning that the model complies with what is known about the system and its outcomes provide a robust basis for decision making.[...]Validation of health economic models should be seen as a critical component of evidence-based decision making in healthcare. However, as of today, it still faces several important challenges, including the lack of consensus guidance and standardised procedures, the need for greater rigour or the question of who should oversee the validation process. To address these challenges, we encourage model developers, agencies requiring models for their decision making and editors of journals that publish models to recommend the use of state-of-the-art tools for reporting (and conducting) validations of health economic models, such as those mentioned in this editorial

Cost recommendation under uncertainty in IQWiG’s efficiency frontier framework

Background: The National Institute for Quality and Efficiency in Health Care (IQWiG) employs an efficiency frontier (EF) framework to facilitate setting maximum reimbursable prices for new interventions. Probabilistic sensitivity analysis (PSA) is used when yes/no reimbursement decisions are sought based on a fixed threshold. In the IQWiG framework, an additional layer of complexity arises as the EF itself may vary its shape in each PSA iteration, and thus the willingness-to-pay, indicated by the EF segments, may vary. Objectives: To explore the practical problems arising when, within the EF approach, maximum reimbursable prices for new interventions are sought through PSA. Methods: When the EF is varied in a PSA, cost recommendations for new interventions may be determined by the mean or the median of the distances between each intervention’s point estimate and each EF. Implications of using these metrics were explored in a simulation study based on the model used by IQWiG to assess the cost-effectiveness of 4 antidepressants. Results. Depending on the metric used, cost recommendations can be contradictory. Recommendations based on the mean can also be inconsistent. Results (median) suggested that costs of duloxetine, venlafaxine, mirtazapine, and bupropion should be decreased by €131, €29, €12, and €99, respectively. These recommendations were implemented and the analysis repeated. New results suggested keeping the costs as they were. The percentage of acceptable PSA outcomes increased 41% on average, and the uncertainty associated to the net health benefit was significantly reduced. Conclusions: The median of the distances between every intervention outcome and every EF is a good proxy for the cost recommendation that would be given should the EF be fixed. Adjusting costs according to the median increased the probability of acceptance and reduced the uncertainty around the net health benefit distribution, resulting in a reduced uncertainty for decision makers

Cost for Treatment of Chronic Lymphocytic Leukemia in Specialized Institutions of Ukraine

AbstractObjectiveThe aim of this study was to identify, from a health care perspective, the cost of treatment for chronic lymphocytic leukemia in specialized hospitals in Ukraine.MethodsCost analysis was performed by using retrospective data between 2006 and 2010 from patient-file databases of two specialized hospitals (145 patients). Uncertainty was assessed by using bootstrapping and multivariate sensitivity analyses. Linear regression analysis was used to analyze whether patients’ characteristics are related to health care costs. In addition, one-way analysis of variance (Welch test) and paired-sample t test were conducted to compare mean costs of treatment between the two hospitals and mean expenses for drugs and in-hospital stay.ResultsThe average annual cost for a patient’s drug treatment is 2047 EUR. The cost of hospitalization was significantly lower (t = 5.026; significance two-tailed = 0.000) and equal to 541 EUR per person, resulting in total expenditures of 2589 EUR. Mean total costs in the bootstrap analysis were equal to 2584 EUR (median 2576 EUR, 97.5th percentile 3223 EUR; 2.5th percentile 1987 EUR). The regression analysis did not reveal a relation between patients’ characteristics and health care costs, although hospital choice was an influential parameter (β = −0.260; significance = 0.002). Significant difference in mean costs of two analyzed hospitals was also confirmed by one-way analysis of variance (Welch statistics 19.222, P = 0.000).ConclusionsDrug treatment comprises the largest portion of total costs, but differences between hospitals exist. Because many patients in Ukraine pay out of pocket for in-hospital drugs, these costs are a high economic burden for patients with chronic lymphocytic leukemia

The increasing importance of a continence nurse specialist to improve outcomes and save costs of urinary incontinence care

__Background:__ In an ageing population, it is inevitable to improve the management of care for community-dwellingelderly with incontinence. A previous study showed that implementation of the Optimum Continence ServiceSpecification (OCSS) for urinary incontinence in community-dwelling elderly with four or more chronic diseasesresults in a reduction of urinary incontinence, an improved quality of life, and lower healt

Cost-effectiveness of adding rituximab to fludarabine and cyclophosphamide for treatment of chronic lymphocytic leukemia in Ukraine

The aim of this study was to assess the cost-effectiveness, from a health care perspective, of adding rituximab to fludarabine and cyclophosphamide scheme (FCR versus FC) for treatment-naïve and refractory/relapsed Ukrainian patients with chronic lymphocytic leukemia. A decision-analytic Markov cohort model with three health states and 1-month cycle time was developed and run within a life time horizon. Data from two multinational, prospective, open-label Phase 3 studies were used to assess patients’ survival. While utilities were generalized from UK data, local resource utilization and disease-associated treatment, hospitalization, and side effect costs were applied. The alternative scenario was performed to assess the impact of lower life expectancy of the general population in Ukraine on the incremental cost-effectiveness ratio (ICER) for treatment-naïve patients. One-way, two-way, and probabilistic sensitivity analyses were conducted to assess the robustness of the results. The ICER (in US dollars) of treating chronic lymphocytic leukemia patients with FCR versus FC is US$8,704 per quality-adjusted life year gained for treatment-naïve patients and US$11,056 for refractory/relapsed patients. When survival data were modified to the lower life expectancy of the general population in Ukraine, the ICER for treatment-naïve patients was higher than US$13,000. This value is higher than three times the current gross domestic product per capita in Ukraine. Sensitivity analyses have shown a high impact of rituximab costs and a moderate impact of differences in utilities on the ICER. Furthermore, probabilistic sensitivity analyses have shown that for refractory/relapsed patients the probability of FCR being cost-effective is higher than for treatment-naïve patients and is close to one if the threshold is higher than US$15,000. State coverage of rituximab treatment may be considered a cost-effective treatment for the Ukrainian population u

AdViSHE: A Validation-Assessment Tool of Health-Economic Models for Decision Makers and Model Users

Background: A trade-off exists between building confidence in health-economic (HE) decision models and the use of scarce resources. We aimed to create a practical tool provid

How to Address Uncertainty in Health Economic Discrete-Event Simulation Models

__Background__. Evaluation of personalized treatment options requires health economic models that include multiple patient characteristics. Patient-level discrete-event simulation (DES) models are deemed appropriate because of their ability to simulate a variety of characteristics and treatment pathways. However, DES models are scarce in the literature, and details about their methods are often missing. __Methods__. We describe 4 challenges associated with modeling heterogeneity and structural, stochastic, and parameter uncertainty that can be encountered during the development of DES models. We explain why these are important and how to correctly implement them. To illustrate the impact of the modeling choices discussed, we use (results of) a model for chronic obstructive pulmonary disease (COPD) as a case study. __Results__. The results from the case study showed that, under a correct implementation of the uncertainty in the model, a hypothetical intervention can be deemed as cost-effective. The consequences of incorrect modeling uncertainty included an increase in the incremental cost-effectiveness ratio ranging from 50% to almost a factor of 14, an extended life expectancy of approximately 1.4 years, and an enormously increased uncertainty around the model outcomes. Thus, modeling uncertainty incorrectly can have substantial implications for decision making. __Conclusions__. This article provides guidance on the implementation of uncertainty in DES models and improves the transparency of reporting uncertainty methods. The COPD case study illustrates the issues described in the article and helps understanding them better. The model R code shows how the uncertainty was implemented. For readers not familiar with R, the model’s pseudo-code can be used to understand how the model works. By doing this, we can help other developers, who are likely to face similar challenges to those described here

A blueprint for health technology assessment capacity building:lessons learned from Malta

Objectives The development and strengthening of health technology assessment (HTA) capacity on the individual and organizational level and the wider environment is relevant for cooperation on HTAs. Based on the Maltese case, we provide a blueprint for building HTA capacity. Methods A set of activities were developed based on Pichler et al.'s framework and the starting HTA capacity in Malta. Individual level activities focused on strengthening epidemiological and health economic skills through online and in-person training. On the organizational level, a new HTA framework was developed which was subsequently utilized in a shadow assessment. Awareness campaign activities raised awareness and support in the wider environment where HTAs are conducted and utilized. Results The time needed to build HTA capacity exceeded the planned two years accommodating the learning progress of the assessors. In addition to the planned trainings, webinars supplemented the online courses, allowing for more knowledge exchange. The advanced online course was extended over time to facilitate learning next to the assessors' daily tasks. Training sessions were added to implement the new economic evaluation framework, which was utilized in a second shadow assessment. Awareness by decision-makers was achieved with reports, posters, and an article on the current and developing HTA capacity. Conclusions It takes time and much (hands-on) training to build skills for conducting complex assessment such as HTAs. Facilitating exchange with knowledgeable parties is crucial for succeeding as well as the buy-in of local managers motivating staff. Decision-makers need to be on-boarded for the continued success of HTA capacity building.</p

A blueprint for health technology assessment capacity building:lessons learned from Malta

Objectives The development and strengthening of health technology assessment (HTA) capacity on the individual and organizational level and the wider environment is relevant for cooperation on HTAs. Based on the Maltese case, we provide a blueprint for building HTA capacity. Methods A set of activities were developed based on Pichler et al.'s framework and the starting HTA capacity in Malta. Individual level activities focused on strengthening epidemiological and health economic skills through online and in-person training. On the organizational level, a new HTA framework was developed which was subsequently utilized in a shadow assessment. Awareness campaign activities raised awareness and support in the wider environment where HTAs are conducted and utilized. Results The time needed to build HTA capacity exceeded the planned two years accommodating the learning progress of the assessors. In addition to the planned trainings, webinars supplemented the online courses, allowing for more knowledge exchange. The advanced online course was extended over time to facilitate learning next to the assessors' daily tasks. Training sessions were added to implement the new economic evaluation framework, which was utilized in a second shadow assessment. Awareness by decision-makers was achieved with reports, posters, and an article on the current and developing HTA capacity. Conclusions It takes time and much (hands-on) training to build skills for conducting complex assessment such as HTAs. Facilitating exchange with knowledgeable parties is crucial for succeeding as well as the buy-in of local managers motivating staff. Decision-makers need to be on-boarded for the continued success of HTA capacity building.</p