Rescaling quality of life values from discrete choice experiments for use as QALYs: a cautionary tale

Background: Researchers are increasingly investigating the potential for ordinal tasks such as ranking and discrete choice experiments to estimate QALY health state values. However, the assumptions of random utility theory, which underpin the statistical models used to provide these estimates, have received insufficient attention. In particular, the assumptions made about the decisions between living states and the death state are not satisfied, at least for some people. Estimated values are likely to be incorrectly anchored with respect to death (zero) in such circumstances. Methods: Data from the Investigating Choice Experiments for the preferences of older people CAPability instrument (ICECAP) valuation exercise were analysed. The values (previously anchored to the worst possible state) were rescaled using an ordinal model proposed previously to estimate QALY-like values. Bootstrapping was conducted to vary artificially the proportion of people who conformed to the conventional random utility model underpinning the analyses. Results: Only 26% of respondents conformed unequivocally to the assumptions of conventional random utility theory. At least 14% of respondents unequivocally violated the assumptions. Varying the relative proportions of conforming respondents in sensitivity analyses led to large changes in the estimated QALY values, particularly for lower-valued states. As a result these values could be either positive (considered to be better than death) or negative (considered to be worse than death). Conclusion: Use of a statistical model such as conditional (multinomial) regression to anchor quality of life values from ordinal data to death is inappropriate in the presence of respondents who do not conform to the assumptions of conventional random utility theory. This is clearest when estimating values for that group of respondents observed in valuation samples who refuse to consider any living state to be worse than death: in such circumstances the model cannot be estimated. Only a valuation task requiring respondents to make choices in which both length and quality of life vary can produce estimates that properly reflect the preferences of all respondents

The use of discrete choice experiments to inform health workforce policy: a systematic review.

BACKGROUND: Discrete choice experiments have become a popular study design to study the labour market preferences of health workers. Discrete choice experiments in health, however, have been criticised for lagging behind best practice and there are specific methodological considerations for those focused on job choices. We performed a systematic review of the application of discrete choice experiments to inform health workforce policy. METHODS: We searched for discrete choice experiments that examined the labour market preferences of health workers, including doctors, nurses, allied health professionals, mid-level and community health workers. We searched Medline, Embase, Global Health, other databases and grey literature repositories with no limits on date or language and contacted 44 experts. Features of choice task and experimental design, conduct and analysis of included studies were assessed against best practice. An assessment of validity was undertaken for all studies, with a comparison of results from those with low risk of bias and a similar objective and context. RESULTS: Twenty-seven studies were included, with over half set in low- and middle-income countries. There were more studies published in the last four years than the previous ten years. Doctors or medical students were the most studied cadre. Studies frequently pooled results from heterogeneous subgroups or extrapolated these results to the general population. Only one third of studies included an opt-out option, despite all health workers having the option to exit the labour market. Just five studies combined results with cost data to assess the cost effectiveness of various policy options. Comparison of results from similar studies broadly showed the importance of bonus payments and postgraduate training opportunities and the unpopularity of time commitments for the uptake of rural posts. CONCLUSIONS: This is the first systematic review of discrete choice experiments in human resources for health. We identified specific issues relating to this application of which practitioners should be aware to ensure robust results. In particular, there is a need for more defined target populations and increased synthesis with cost data. Research on a wider range of health workers and the generalisability of results would be welcome to better inform policy

Experimental Design

AbstractThis chapter covers various issues related to the experimental design, a statistical technique at the core of a discrete choice experiment. Specifically, it focuses on the dimensionality of a choice experiment and the statistical techniques used to allocate attribute levels to choice tasks. Among others, the pros and cons of orthogonal designs, optimal orthogonal in the differences designs as well as efficient designs are addressed. The last section shows how a simulation exercise can help to test the appropriateness of the experimental design

Discrete Choice Experiments: A Guide to Model Specification, Estimation and Software

We provide a user guide on the analysis of data (including best–worst and best–best data) generated from discrete-choice experiments (DCEs), comprising a theoretical review of the main choice models followed by practical advice on estimation and post-estimation. We also provide a review of standard software. In providing this guide, we endeavour to not only provide guidance on choice modelling but to do so in a way that provides a ‘way in’ for researchers to the practicalities of data analysis. We argue that choice of modelling approach depends on the research questions, study design and constraints in terms of quality/quantity of data and that decisions made in relation to analysis of choice data are often interdependent rather than sequential. Given the core theory and estimation of choice models is common across settings, we expect the theoretical and practical content of this paper to be useful to researchers not only within but also beyond health economics

Interactive stated choice surveys: a study of air travel behaviour

Stated preference (SP) experiments are becoming an increasingly popular survey methodology for investigating travel behaviour. Nevertheless, some evidence suggests that SP experiments do not mirror decisions in real markets. With an increasing number of real world decisions made using the internet, an opportunity exists to improve the realism of the SP counterparts of such choices by aligning the choice environment with such online portals. In this paper, we illustrate the benefits of such an approach in the context of air travel surveys. Our survey is modelled on the interface and functionality of an online travel agent (OTA). As with a real OTA, many ticket options are presented. Sort tools allow the options to be reordered, search tools allow options to be removed from consideration, and a further tool allows attributes to be hidden and shown. Extensive use of these tools is made by the 462 respondents. A traditional SP component was also completed by the respondents. Our exploratory analysis as well as random utility model estimation results confirm not only that respondents seem to engage more actively with the interactive survey, but also that the resulting data allows for better performance in model estimation compared to a more conventional SP experiment. These results have implications for the study of other complex travel choices where interactive surveys may similarly be preferable to standard approaches

Discrete choice modelling : basic principles and application to parking policy assessment

No abstract