Assessing the impact of health technology assessment in the Netherlands

Copyright © Cambridge University Press 2008Objectives: Investments in health research should lead to improvements in health and health care. This is also the remit of the main HTA program in the Netherlands. The aims of this study were to assess whether the results of this program have led to such improvements and to analyze how best to assess the impact from health research.Methods: We assessed the impact of individual HTA projects by adapting the "payback framework" developed in the United Kingdom. We conducted dossier reviews and sent a survey to principal investigators of forty-three projects awarded between 2000 and 2003. We then provided an overview of documented output and outcome that was assessed by ten HTA experts using a scoring method. Finally, we conducted five case studies using information from additional dossier review and semistructured key informant interviews.Results: The findings confirm that the payback framework is a useful approach to assess the impact of HTA projects. We identified over 101 peer reviewed papers, more than twenty-five PhDs, citations of research in guidelines (six projects), and implementation of new treatment strategies (eleven projects). The case studies provided greater depth and understanding about the levels of impact that arise and why and how they have been achieved.Conclusions: It is generally too early to determine whether the HTA program led to actual changes in healthcare policy and practice. However, the results can be used as a baseline measurement for future evaluation and can help funding organizations or HTA agencies consider how to assess impact, possibly routinely. This, in turn, could help inform research strategies and justify expenditure for health research.This research is funded by ZonMw, the Netherlands organization for health research and development (project 945-15-001)

A review of health utilities using the EQ-5D in studies of cardiovascular disease

RIGHTS : This article is licensed under the BioMed Central licence at http://www.biomedcentral.com/about/license which is similar to the 'Creative Commons Attribution Licence'. In brief you may : copy, distribute, and display the work; make derivative works; or make commercial use of the work - under the following conditions: the original author must be given credit; for any reuse or distribution, it must be made clear to others what the license terms of this work are.Abstract Background The EQ-5D has been extensively used to assess patient utility in trials of new treatments within the cardiovascular field. The aims of this study were to review evidence of the validity and reliability of the EQ-5D, and to summarise utility scores based on the use of the EQ-5D in clinical trials and in studies of patients with cardiovascular disease. Methods A structured literature search was conducted using keywords related to cardiovascular disease and EQ-5D. Original research studies of patients with cardiovascular disease that reported EQ-5D results and its measurement properties were included. Results Of 147 identified papers, 66 met the selection criteria, with 10 studies reporting evidence on validity or reliability and 60 reporting EQ-5D responses (VAS or self-classification). Mean EQ-5D index-based scores ranged from 0.24 (SD 0.39) to 0.90 (SD 0.16), while VAS scores ranged from 37 (SD 21) to 89 (no SD reported). Stratification of EQ-5D index scores by disease severity revealed that scores decreased from a mean of 0.78 (SD 0.18) to 0.51 (SD 0.21) for mild to severe disease in heart failure patients and from 0.80 (SD 0.05) to 0.45 (SD 0.22) for mild to severe disease in angina patients. Conclusions The published evidence generally supports the validity and reliability of the EQ-5D as an outcome measure within the cardiovascular area. This review provides utility estimates across a range of cardiovascular subgroups and treatments that may be useful for future modelling of utilities and QALYs in economic evaluations within the cardiovascular area.Published versio

Mapping of the EQ-5D index from clinical outcome measures and demographic variables in patients with coronary heart disease.

RIGHTS : This article is licensed under the BioMed Central licence at http://www.biomedcentral.com/about/license which is similar to the 'Creative Commons Attribution Licence'. In brief you may : copy, distribute, and display the work; make derivative works; or make commercial use of the work - under the following conditions: the original author must be given credit; for any reuse or distribution, it must be made clear to others what the license terms of this work are.BACKGROUND: The EuroQoL 5D (EQ-5D) is a questionnaire that provides a measure of utility for cost-effectiveness analysis. The EQ-5D has been widely used in many patient groups, including those with coronary heart disease. Studies often require patients to complete many questionnaires and the EQ-5D may not be gathered. This study aimed to assess whether demographic and clinical outcome variables, including scores from a disease specific measure, the Seattle Angina Questionnaire (SAQ), could be used to predict, or map, the EQ-5D index value where it is not available. METHODS: Patient-level data from 5 studies of cardiac interventions were used. The data were split into two groups - approximately 60% of the data were used as an estimation dataset for building models, and 40% were used as a validation dataset. Forward ordinary least squares linear regression methods and measures of prediction error were used to build a model to map to the EQ-5D index. Age, sex, a proxy measure of disease stage, Canadian Cardiovascular Society (CCS) angina severity class, treadmill exercise time (ETT) and scales of the SAQ were examined. RESULTS: The exertional capacity (ECS), disease perception (DPS) and anginal frequency scales (AFS) of the SAQ were the strongest predictors of the EQ-5D index and gave the smallest root mean square errors. A final model was chosen with age, gender, disease stage and the ECS, DPS and AFS scales of the SAQ. ETT and CCS did not improve prediction in the presence of the SAQ scales. Bland-Altman agreement between predicted and observed EQ-5D index values was reasonable for values greater than 0.4, but below this level predicted values were higher than observed. The 95% limits of agreement were wide (-0.34, 0.33). CONCLUSIONS: Mapping of the EQ-5D index in cardiac patients from demographics and commonly measured cardiac outcome variables is possible; however, prediction for values of the EQ-5D index below 0.4 was not accurate. The newly designed 5-level version of the EQ-5D with its increased ability to discriminate health states may improve prediction of EQ-5D index values

Relationship between the EQ-5D index and measures of clinical outcomes in selected studies of cardiovascular interventions.

RIGHTS : This article is licensed under the BioMed Central licence at http://www.biomedcentral.com/about/license which is similar to the 'Creative Commons Attribution Licence'. In brief you may : copy, distribute, and display the work; make derivative works; or make commercial use of the work - under the following conditions: the original author must be given credit; for any reuse or distribution, it must be made clear to others what the license terms of this work are.BACKGROUND: The EuroQoL 5D (EQ-5D) has been widely used in studies of cardiac disease, but its measurement properties in this group are not well established. The study aimed to quantify the relationship between measures commonly used in studies of cardiac disease and the EQ-5D index across different levels of disease severity. METHODS: Patient-level data from 7 studies of cardiac interventions were used, which included randomised trials and observational studies. Relationships between the EQ-5D index and commonly used cardiac measures, Canadian Cardiovascular Society (CCS) angina severity class, treadmill exercise time (ETT) and scales of the Seattle Angina Questionnaire (SAQ) were examined. Mixed effects linear regression was used to assess these relationships, with the EQ-5D index as the response. RESULTS: Study sample sizes ranged from 68 to 2419. Mean baseline EQ-5D index ranged from 0.77 in patients at diagnosis (95% CI 0.75, 0.78) to 0.43 in patients with advanced disease (95% CI 0.39, 0.48) and differed significantly across studies (p < 0.001). There was evidence of a ceiling effect in patients at diagnosis. The minimum clinically important difference of a one minute increase in ETT was associated with a 0.019 (95% CI 0.014, 0.025) increase in EQ-5D index. One class increase in CCS was associated with a 0.11 (95% CI 0.09, 0.13) decrease in EQ-5D index. A 10 unit increase in SAQ scales was associated with increases between 0.04 and 0.07 in EQ-5D index (95% CIs 0.03, 0.05 and 0.05, 0.08). Tests of heterogeneity indicated the EQ-5D-covariate relationships were consistent across levels of disease severity for ETT and the treatment satisfaction scale of the SAQ, but heterogeneous for age, gender, CCS angina class and other scales of the SAQ. CONCLUSION: The EQ-5D index varies with coronary disease severity. The relationship between the EQ-5D index and an outcome measure used in cardiac intervention studies, ETT, was consistent across disease severity levels, but the relationship between demographic variables, CCS angina class and most of the SAQ scales and the EQ-5D index was heterogeneous for patients with different levels of coronary disease. Differences in the EQ-5D index associated with clinically important differences in cardiac measures can be quantified and vary between three important examples - angina class, ETT and SAQ

Exploring what lies behind public preferences for avoiding health losses caused by lapses in healthcare safety and patient lifestyle choices

© 2013 Singh et al.; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0),which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.This article has been made available through the Brunel Open Access Publishing Fund.Background: Although many studies have identified public preferences for prioritising health care interventions based on characteristics of recipient or care, very few of them have examined the reasons for the stated preferences. We conducted an on-line person trade-off (PTO) study (N=1030) to investigate whether the public attach a premium to the avoidance of ill health associated with alternative types of responsibilities: lapses in healthcare safety, those caused by individual action or lifestyle choice; or genetic conditions. We found that the public gave higher priority to prevention of harm in a hospital setting such as preventing hospital associated infections than genetic disorder but drug administration errors were valued similar to genetic disorders. Prevention of staff injuries, lifestyle diseases and sports injuries, were given lower priority. In this paper we aim to understand the reasoning behind the responses by analysing comments provided by respondents to the PTO questions. Method: A majority of the respondents who participated in the survey provided brief comments explaining preferences in free text responses following PTO questions. This qualitative data was transformed into explicit codes conveying similar meanings. An overall coding framework was developed and a reliability test was carried out. Recurrent patterns were identified in each preference group. Comments which challenged the assumptions of hypothetical scenarios were also investigated. Results: NHS causation of illness and a duty of care were the most cited reasons to prioritise lapses in healthcare safety. Personal responsibility dominated responses for lifestyle related contexts, and many respondents mentioned that health loss was the result of the individual’s choice to engage in risky behaviour. A small proportion of responses questioned the assumptions underlying the PTO questions. However excluding these from the main analysis did not affect the conclusions. Conclusion: Although some responses indicated misunderstanding or rejection of assumptions we put forward, the results were still robust. The reasons put forward for responses differed between comparisons but responsibility was the most frequently cited. Most preference elicitation studies only focus on eliciting numerical valuations but allowing for qualitative data can augment understanding of preferences as well as verifying results.EPSRC through the MATCH programme(EP/F063822/1 and EP/G012393/1) and HERG within Brunel University

Illustrating potential efficiency gains from using cost-effectiveness evidence to reallocate Medicare expenditures

This article is available open access through the publisher’s website at the linke below. Copyright @ 2013, International Society for Pharmacoeconomics and Outcomes Research (ISPOR).This article has been made available through the Brunel Open Access Publishing Fund.Objectives - The Centers for Medicare & Medicaid Services does not explicitly use cost-effectiveness information in national coverage determinations. The objective of this study was to illustrate potential efficiency gains from reallocating Medicare expenditures by using cost-effectiveness information, and the consequences for health gains among Medicare beneficiaries. Methods - We included national coverage determinations from 1999 through 2007. Estimates of cost-effectiveness were identified through a literature review. For coverage decisions with an associated cost-effectiveness estimate, we estimated utilization and size of the “unserved” eligible population by using a Medicare claims database (2007) and diagnostic and reimbursement codes. Technology costs originated from the cost-effectiveness literature or were estimated by using reimbursement codes. We illustrated potential aggregate health gains from increasing utilization of dominant interventions (i.e., cost saving and health increasing) and from reallocating expenditures by decreasing investment in cost-ineffective interventions and increasing investment in relatively cost-effective interventions. Results - Complete information was available for 36 interventions. Increasing investment in dominant interventions alone led to an increase of 270,000 quality-adjusted life-years (QALYs) and savings of $12.9 billion. Reallocation of a broader array of interventions yielded an additional 1.8 million QALYs, approximately 0.17 QALYs per affected Medicare beneficiary. Compared with the distribution of resources prior to reallocation, following reallocation a greater proportion was directed to oncology, diagnostic imaging/tests, and the most prevalent diseases. A smaller proportion of resources went to cardiology, treatments (including drugs, surgeries, and medical devices, as opposed to nontreatments such as preventive services), and the least prevalent diseases. Conclusions - Using cost-effectiveness information has the potential to increase the aggregate health of Medicare beneficiaries while maintaining existing spending levels.The Commonwealth Fun

The information sources and journals consulted or read by UK paediatricians to inform their clinical practice and those which they consider important: a questionnaire survey

Background: Implementation of health research findings is important for medicine to be evidence-based. Previous studies have found variation in the information sources thought to be of greatest importance to clinicians but publication in peer-reviewed journals is the traditional route for dissemination of research findings. There is debate about whether the impact made on clinicians should be considered as part of the evaluation of research outputs. We aimed to determine first which information sources are generally most consulted by paediatricians to inform their clinical practice, and which sources they considered most important, and second, how many and which peer-reviewed journals they read. Methods: We enquired, by questionnaire survey, about the information sources and academic journals that UK medical paediatric specialists generally consulted, attended or read and considered important to their clinical practice. Results: The same three information sources – professional meetings & conferences, peerreviewed journals and medical colleagues – were, overall, the most consulted or attended and ranked the most important. No one information source was found to be of greatest importance to all groups of paediatricians. Journals were widely read by all groups, but the proportion ranking them first in importance as an information source ranged from 10% to 46%. The number of journals read varied between the groups, but Archives of Disease in Childhood and BMJ were the most read journals in all groups. Six out of the seven journals previously identified as containing best paediatric evidence are the most widely read overall by UK paediatricians, however, only the two most prominent are widely read by those based in the community. Conclusion: No one information source is dominant, therefore a variety of approaches to Continuing Professional Development and the dissemination of research findings to paediatricians should be used. Journals are an important information source. A small number of key ones can be identified and such analysis could provide valuable additional input into the evaluation of clinical research outputs

Understanding factors associated with the translation of cardiovascular research: A multinational case study approach

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited.This article has been made available through the Brunel Open Access Publishing Fund.Background: Funders of health research increasingly seek to understand how best to allocate resources in order to achieve maximum value from their funding. We built an international consortium and developed a multinational case study approach to assess benefits arising from health research. We used that to facilitate analysis of factors in the production of research that might be associated with translating research findings into wider impacts, and the complexities involved. Methods: We built on the Payback Framework and expanded its application through conducting co-ordinated case studies on the payback from cardiovascular and stroke research in Australia, Canada and the United Kingdom. We selected a stratified random sample of projects from leading medical research funders. We devised a series of innovative steps to: minimize the effect of researcher bias; rate the level of impacts identified in the case studies; and interrogate case study narratives to identify factors that correlated with achieving high or low levels of impact. Results: Twenty-nine detailed case studies produced many and diverse impacts. Over the 15 to 20 years examined, basic biomedical research has a greater impact than clinical research in terms of academic impacts such as knowledge production and research capacity building. Clinical research has greater levels of wider impact on health policies, practice, and generating health gains. There was no correlation between knowledge production and wider impacts. We identified various factors associated with high impact. Interaction between researchers and practitioners and the public is associated with achieving high academic impact and translation into wider impacts, as is basic research conducted with a clinical focus. Strategic thinking by clinical researchers, in terms of thinking through pathways by which research could potentially be translated into practice, is associated with high wider impact. Finally, we identified the complexity of factors behind research translation that can arise in a single case. Conclusions: We can systematically assess research impacts and use the findings to promote translation. Research funders can justify funding research of diverse types, but they should not assume academic impacts are proxies for wider impacts. They should encourage researchers to consider pathways towards impact and engage potential research users in research processes. © 2014 Wooding et al.; licensee BioMed Central Ltd.RAND Europe and HERG, with subsequent funding from the NHFA, the HSFC and the CIHR. This research was also partially supported by the Policy Research Programme in the English Department of Health

The demand for sports and exercise: Results from an illustrative survey

Funding from the Department of Health policy research programme was used in this study.There is a paucity of empirical evidence on the extent to which price and perceived benefits affect the level of participation in sports and exercise. Using an illustrative sample of 60 adults at Brunel University, West London, we investigate the determinants of demand for sports and exercise. The data were collected through face-to-face interviews that covered indicators of sports and exercise behaviour; money/time price and perceived benefits of participation; and socio- economic/demographic details. Count, linear and probit regression models were fitted as appropriate. Seventy eight per cent of the sample participated in sports and exercise and spent an average of £27 per month and an average of 20 min travelling per occasion of sports and exercise. The demand for sport and exercise was negatively associated with time (travel or access time) and ‘variable’ price and positively correlated with ‘fixed’ price. Demand was price inelastic, except in the case of meeting the UK government’s recommended level of participation, which is time price elastic (elasticity = −2.2). The implications of data from a larger nationally representative sample as well as the role of economic incentives in influencing uptake of sports and exercise are discussed.This article is available through the Brunel Open Access Publishing Fund