Skip to main content
Article thumbnail
Location of Repository

Dealing with uncertainty in the economic evaluation of health care technologies

By Susan Griffin


This collection of papers aims to addresses why, and how, uncertainty should be incorporated within economic evaluations of health care technologies and how the results can be used to inform decision making. Section one provides a brief overview of the methodological and policy background of using economic evaluation to inform decisions about health care technologies. With the context for utilising economic evaluation to inform reimbursement decisions established, sources of uncertainty are considered at each stage of the evaluation. Section two discusses the identification of relevant evidence to inform estimates of the effectiveness of health care technologies. The key uncertainty addressed is how to assess whether the available evidence can be analysed to provide an unbiased estimate of the treatment effect. As multiple sources of evidence may exist, Section three explains how they can be combined to sum up the available evidence, and in Section four the techniques for characterising uncertainty within that combined analysis are examined. Sections five and six demonstrate why a characterisation of uncertainty is crucial in order to inform decisions about the need for further research and the consequences of making reimbursement decisions under uncertainty. Methods to establish the opportunity cost of uncertainty and sufficiency of the evidence base are reviewed in Section five with a view to informing decisions to acquire further research. In Section six the irreversible aspects of reimbursement decisions, as well as the interdependence of decisions about reimbursement and about further research is illustrated. These factors spell out why an economic evaluation that does not deal with uncertainty will be inadequate for informing reimbursement decisions. Finally, Section seven concludes the thesis and summarises the contribution of the presented papers and areas for further research

Publisher: Health Sciences (York)
Year: 2010
OAI identifier:

Suggested articles


  1. (2007). A Critique and Impact Analysis of Decision Modelling Assumptions. Medical Decision Making
  2. A method for the detailed assessment of the appropriateness of medical technologies.
  3. A pilot study of Value of Information analysis to support research recommendations for the National Institute for Clinical Excellence. York: Centre for Health Economics. Research Paper 4., 2005:Chapter 4.
  4. (2003). A pilot study on the use of decision theory and value of information analysis as part of the NHS health technology assessment programme. Health Technology Assessment
  5. A review of studies mapping (or cross walking) non-preference based measures of health to generic preference-based measures.
  6. A review of techniques for parameter sensitivity analysis of environmental models. Environmental monitoring and assessment 1994;32(2):135-154.
  7. (2006). A review of UK health research funding. London: HM Treasury,
  8. A systematic review of the clinical effectiveness and cost effectiveness of topotecan, pegylated liposomal doxorubicin hydrochloride and paclitaxel for second-line or subsequent treatment of advanced ovarian cancer (up-date).
  9. A taxonomy of model structures for economic evaluation of health technologies.
  10. A Utility Maximization Model for Evaluation of Health Care Programs.
  11. Access with Evidence Development Schemes: A Framework for Description and Evaluation Pharmacoeconomics
  12. Adjusted indirect comparison may be less biased than direct comparison for evaluating new pharmaceutical interventions.
  13. Advanced methods in meta-analysis: multivariate approach and meta-regression.
  14. (1996). An economic approach to clinical trial design and research priority setting.
  15. (1985). An Examination of Response-Surface Methodologies for Uncertainty Analysis in Assessment Models.
  16. (1988). An Investigation of Uncertainty and Sensitivity Analysis Techniques for Computer Models. Risk Analysis
  17. (1997). Analysis of Incomplete Multivariate Data.
  18. (1961). Applied Statistical Decision Theory: Division of Research,
  19. (2000). Assessing Quality in Decision Analytic CostEffectiveness Models: A Suggested Framework and Example of Application Pharmacoeconomics
  20. (1995). Assessment and Propogation of Model Uncertainty.
  21. (2006). Bayesian Methods for Evidence Synthesis in Cost-Effectiveness Analysis Pharmacoeconomics
  22. Bayesian methods in meta-analysis and evidence synthesis.
  23. (1999). Bayesian Model Averaging: A Tutorial.
  24. Bivariate random-effects meta-analysis and the estimation of between-study correlation.
  25. (1996). Borrowing strength from external trials in a meta-analysis. Statistics in Medicine
  26. (2007). Calculating Partial Expected Value of Perfect Information via Monte Carlo Sampling Algorithms. Medical Decision Making
  27. Can we estimate the 'scoial' value of a QALY? Four core issues to resolve.
  28. Can't Get No Satisfaction? Will Pay for Performance Help?: Toward an Economic Framework for Understanding Performance-Based RiskSharing Agreements for Innovative Medical Products.
  29. Characterizing structural uncertainty in decision-analytic models: a review and application of methods.
  30. Choice of modelling techniques for evaluating health care interventions.
  31. Combination of direct and indirect evidence in mixed treatment comparisons.
  32. (2008). Comparing the Incomparable? A Systematic Review of Competing Techniques for Converting Descriptive Measures of Health Status into QALYWeights. Medical Decision Making
  33. Continuing the multiple sclerosis risk sharing scheme is unjustified
  34. Converting condition-specific measures into preference-based outcomes for use in economic evaluation.
  35. Converting the SF-12 into the EQ-5D: An Empirical Comparison of Methodologies.
  36. Cost effectiveness of clinically appropriate decisions on alternative treatments for angina pectoris: prospective observational study.
  37. (1992). Cost effectiveness/utility analyses. Do current decision rules lead us to where we want to be?
  38. Cost-benefit analysis: bastard science? And/or insidious poison in the body politick?
  39. Coverage with evidence development: an examination of conceptual and policy issues.
  40. Criteria for evaluating evidence on public health interventions.
  41. Dangerous Omissions: The Consequences of Ignoring Decision Uncertainty. Health Economics 2010;Published online in Wiley Interscience.
  42. (1988). Decision Analysis: Practice and Promise.
  43. Decision Analytic Modelling in the Economic Evaluation of Health Technologies: A Consensus Statement.
  44. (2006). Decision Modelling for Health Economic Evaluation.
  45. Decision-Aiding Models: Validation, Assessment, and Related Issues for Policy Analysis.
  46. (1982). Designing evaluations of educational and social programes.
  47. (2008). Dimensions of design space: a decision-theoretic approach to optimal research design. York: Centre for Health Economics,
  48. (1998). Econometrics in outcomes research: the use of instrumental variables.
  49. (1996). Economic evaluation and health care decision-making. Health Policy
  50. (2009). Economic evaluation of public health interventions.
  51. Efficiency, equity and budgetary policies: informing decisions using mathematical programming.
  52. (2005). Establishing the Cost-Effectiveness of New Pharmaceuticals uncer Conditions of Uncertainty - When Is There Sufficient Evidence? Value Health
  53. Estimating the causal effects of treatments in randomized and nonrandomized studies.
  54. Estimating the expected value of partial perfect information: a review of methods.
  55. Evaluation of health technology: Economic issues for health policy and policy issues for economic appraisal.
  56. Evaluation of networks of randomized trials.
  57. (2008). Evidence-Based Decision Making: When Should We Wait For More Information? Health Affairs
  58. Evidence-based health policy: context and utilisation.
  59. (1996). Evidence-based medicine: what it is and what it isn't.
  60. Evidence-Based Public Health: Moving Beyond Randomized Trials.
  61. Expected value of information and decision making in HTA.
  62. (2004). Expected value of Sample Information Calculations in Medical Decision Modeling. Medical Decision Making
  63. Expected value of sample information for Weibull survival data.
  64. (1963). Experimental and quasi-experimental designs for research.
  65. Exploring the research decision space: the expected value of information for sequential research designs Medical Decision Making
  66. Exploring Uncertainty in Cost-effectiveness Analysis.
  67. Factors relevant to the validity of experiments in social settings.
  68. Generalisability in economic evaluation studies in healthcare: a review and case studies.
  69. Globally optimal trial design for local decision making.
  70. (1996). Going beyond a single number: Using probabilistic risk assessment to improve risk management. Human and Ecological Risk Assessment
  71. (2008). Guide to the Methods of Technology Appraisal. London: National Institute for Health and Clinical Excellence,
  72. (1993). Guidelines for the adoption of new technologies: a prescription for uncontrolled growth in expenditures and how to avoid the problem.
  73. Health economic guidelines - similarities, differences and some implications.
  74. (2010). Health. The NHS Constitution: the NHS belongs to us all. London: Department of Health,
  75. Hierarchical models in generalized synthesis of evidence: an example based on studies of breast cancer screening.
  76. How should meta-regression analyses be undertaken and interpreted.
  77. Incorporating direct and indirect evidence using Bayesian methods: an applied case study in ovarian cancer.
  78. Incorporating option values into the economic evaluation of health care technologies.
  79. (2009). Indirect Evidence: Indirect Treatment Comparisons in Meta-Analysis. Ottawa: Canadian Agency for Drugs and Technologies in Health,
  80. Inference and decision at the bedside.
  81. (2008). Latin Hypercube Sampling. In:
  82. Linking payment to health outcomes: A taxonomy and examination of performance-based reimbursement schemes between healthcare payers and manufacturers.
  83. Making a decision to wait for more evidence: when the National Institute for Health and Clinical Excellence recommends a technology only in the context of research.
  84. Mark versus Luke? Appropriate methods for the evaluation of public health interventions.
  85. Markov Chain Monte Carlo estimation of a multi-parameter decision model: consistency of evidence and the accurate assessment of uncertainty.
  86. Mathematical programming for the efficient allocation of health care resources.
  87. Measurement of health state utilities for economic appraisal.
  88. (1998). Meta-analysis of Multitreatment Studies.
  89. (2005). Methods for the economic evaluation of health care programmes. Third ed. Oxford:
  90. Monte Carlo probabilistic sensitivity analysis for patient level simulation models: efficient estimation of mean and variance using ANOVA.
  91. Multiparameter evidence synthesis in epidemiology and medical decision-making: current approaches.
  92. (1987). Multiple Imputation for Nonresponse in Surveys.
  93. (1999). Multiple imputation of missing blood pressure covariates in survival analysis.
  94. (1999). Multiple Imputation: A Primer.
  95. Multiple sclerosis risk sharing scheme: a costly failure.
  96. NHS goes to the PROMS.
  97. On the decision rules of cost-effectiveness analysis.
  98. On the economic foundations of CEA. Ladies and gentlemen, take your positions!
  99. Opiates for the Matches: Matching Methods for Causal Inference.
  100. Optimal clinical trial design using value of information methods with imperfect implementation.
  101. (2004). Pharacoeconomic guidlines around the world. ISPOR connections 2004;Aug 15
  102. Pitfalls in modeling. In:
  103. Principles of Good Practice for Decision Analytic Modeling
  104. Probabilistic analysis and computationally expensive models: necessary and required?
  105. (1986). Probabilistic Analysis of Decision Trees Using Monte Carlo Simulation. Medical Decision Making
  106. Probabilistic sensitivity analysis for NICE technology assessment: not an optional extra.
  107. (1985). Probabilistic sensitivity analysis using Monte Carlo simulation. A practical approach. Medical Decision Making
  108. (1959). Probability and Statistics for Business Decisions.
  109. (1994). Propagation of uncertainty in risk assessments: the need to distinguish between uncertainty due to lack of knowledge and uncertainty due to variability. Risk Analysis
  110. (2006). Publication Bias in Meta-Analysis: Prevention, Assessment and Adjustments.
  111. (1997). Pulling cost-effectiveness analysis up by its bootstraps: a non-parametric approach to confidence interval estimation.
  112. (1999). Rating the appropriateness of coronary angiography, coronary angioplasty and coronary artery bypass grafting: the ACRE study.
  113. Regression methods for covariate adjustment and subgroup analysis for non-censored cost-effectiveness data.
  114. Representing uncertainty: the role of costeffectiveness acceptability curves.
  115. Research Prioritisation Based on Expected Value of Partial Perfect Information: a Case Study on Interventions to Increase Uptake of Breast Cancer Screening.
  116. Review of guidelines for good practice in decision-analytic modelling in health technology assessment.
  117. Sample selection bias as a specification error.
  118. Searching for cost effectiveness thresholds in the NHS.
  119. secondary and meta-analysis of research.
  120. (2004). Securing good health for the whole population. London: HM Treasury,
  121. (1998). Sensitivity analysis and the expected value of perfect information.
  122. Setting priorities for research.
  123. (1992). Spiderplots versus Tornado Diagrams for Sensitivity Analysis. Interfaces
  124. (1995). Statistical decision theory. Cambridge M:
  125. Systematic reviews of trials and other studies.
  126. (1994). Systematic Reviews: Rationale for systematic reviews.
  127. The arrangement of field experiments.
  128. (1990). The case for reassessment of health care technology: once is not enough.
  129. The causes and effects of socio-demographic exclusions from clinical trials.
  130. The central role of propensity score in observational studies for causal effects.
  131. The cost-benefit approach.
  132. (1996). The EuroQoL instrument: an index of health-related quality of life.
  133. The expected value of information and optimal clinical trial design.
  134. (2008). The half-life of truth: appropriate time horizons for research decisions? Medical Decision Making
  135. (1999). The irrelevance of inference: a decision-making approach to the stochastic evaluation of health care technologies.
  136. (1997). The iterative use of economic evaluation as part of the process of health technology assessment.
  137. The NICE Cost-effectiveness Threshold. What it is and What that Means.
  138. (1989). The normative economics of health care finance and provision. Oxford Review of Economic Policy
  139. The option value of delay in health technology assessment.
  140. (1979). The Principles of Practical Cost-Benefit Analysis.
  141. (1997). The results of direct and indirect treatment comparisons in meta-analysis of randomized controlled trials.
  142. The role of modelling in prioritising and planning clinical trials.
  143. The Value of Implementation and the Value of Information: Combined and Uneven Development. Medical Decision Making
  144. (1999). The Value of the Value of Information. Human and Ecological Risk Assessment
  145. The value of value of information: best informing research design and prioritization using current methods.
  146. Tiao GC. Bayesian inference in statistical analysis.
  147. Time and EVSI wait for no patient.
  148. (2006). Uncertain Judgements: Eliciting Expert Probabilities.
  149. Uncertainty and Patient Heterogeneity in Medical Decision Models. Medical Decision Making
  150. Uncertainty and validation of health economic decision models.
  151. (1989). Utilities and quality-adjusted life years.
  152. (1977). Validation of Regression Models: Methods and Examples.
  153. (2004). Value of information literature analysis: a review of applications in health risk management. Medical Decision Making
  154. Whither trial-based economic evaluation for health care decision making?
  155. (1996). Why we need observational studies to evaluate the effectiveness of health care.

To submit an update or takedown request for this paper, please submit an Update/Correction/Removal Request.