Cluster randomized trials: Another problem for cost-effectiveness ratios

Objectives: This work has investigated under what conditions cost-effectiveness data from a cluster randomized trial (CRT) are suitable for analysis using a cluster-adjusted nonparametric bootstrap. The bootstrap's main advantages are in dealing with skewed data and its ability to take correlations between costs and effects into account. However, there are known theoretical problems with a commonly used cluster bootstrap procedure, and the practical implications of these require investigation. Methods: Simulations were used to estimate the coverage of confidence intervals around incremental cost-effectiveness ratios from CRTs using two bootstrap methods. Results: The bootstrap gave excessively narrow confidence intervals, but there was evidence to suggest that, when the number of clusters per treatment arm exceeded 24, it might give acceptable results. The method that resampled individuals as well as clusters did not perform well when cost and effectiveness data were correlated. Conclusions: If economic data from such trials are to be analyzed adequately, then there is a need for further investigations of more complex bootstrap procedures. Similarly, further research is required on methods such as the net benefit approach. Copyright © 2005 Cambridge University Press

Probabilistic models of set-dependent and attribute-level best-worst choice

We characterize a class of probabilistic choice models where the choice probabilities depend on two scales, one with a value for each available option and the other with a value for the set of available options. Then, we develop similar results for a task in which a person is presented with a profile of attributes, each at a pre-specified level, and chooses the best or the best and the worst of those attribute-levels. The latter design is an important variant on previous designs using best-worst choice to elicit preference information, and there is various evidence that it yields reliable interpretable data. Nonetheless, the data from a single such task cannot yield separate measures of the "importance" of an attribute and the "utility" of an attribute-level. We discuss various empirical designs, involving more than one task of the above general type, that may allow such separation of importance and utility. © 2008 Elsevier Inc. All rights reserved

Discrete choice experiments are not conjoint analysis

We briefly review and discuss traditional conjoint analysis (CA) and discrete choice experiments (DCEs), widely used stated preference elicitation methods in several disciplines. We pay particular attention to the origins and basis of CA, and show that it is generally inconsistent with economic demand theory, and is subject to several logical inconsistencies that make it unsuitable for use in applied economics, particularly welfare and policy assessment. We contrast this with DCEs that have a long-standing, well-tested theoretical basis in random utility theory, and we show why and how DCEs are more general and consistent with economic demand theory. Perhaps the major message, though, is that many studies that claim to be doing conjoint analysis are really doing DCE

What do people value when they provide unpaid care for an older person? A meta-ethnography with interview follow-up

Government policies to shift care into the community and demographic changes mean that unpaid (informal) carers will increasingly be relied on to deliver care, particularly to older people. As a result, careful consideration needs to be given to informal care in economic evaluations. Current methods for economic evaluations may neglect important aspects of informal care. This paper reports the development of a simple measure of the caring experience for use in economic evaluations. A meta-ethnography was used to reduce qualitative research to six conceptual attributes of caring. Sixteen semi-structured interviews were then conducted with carers of older people, to check the attributes and develop them into the measure. Six attributes of the caring experience comprise the final measure: getting on, organisational assistance, social support, activities, control, and fulfilment. The final measure (the Carer Experience Scale) focuses on the process of providing care, rather than health outcomes from caring. Arguably this provides a more direct assessment of carers' welfare. Following work to test and scale the measure, it may offer a promising way of incorporating the impact on carers in economic evaluations. © 2008 Elsevier Ltd. All rights reserved

Valuing the ICECAP capability index for older people

This paper reports the first application of the capabilities approach to the development and valuation of an instrument for use in the economic evaluation of health and social care interventions. The ICECAP index of capability for older people focuses on quality of life rather than health or other influences on quality of life, and is intended to be used in decision making across health and social care in the UK. The measure draws on previous qualitative work in which five conceptual attributes were developed: attachment, security, role, enjoyment and control. This paper details the innovative use within health economics of further iterative qualitative work in the UK among 19 informants to refine lay terminology for each of the attributes and levels of attributes used in the eventual index. For the first time within quality of life measurement for economic evaluation, a best-worst scaling exercise has been used to estimate general population values (albeit for the population of those aged 65+ years) for the levels of attributes, with values anchored at one for full capability and zero for no capability. Death was assumed to be a state in which there is no capability. The values obtained indicate that attachment is the attribute with greatest impact but all attributes contribute to the total estimation of capability. Values that were estimated are feasible for use in practical applications of the index to measure the impact of health and social care interventions. © 2008 Elsevier Ltd. All rights reserved

Use of the bootstrap in analysing cost data from cluster randomised trials: some simulation results

BACKGROUND: This work has investigated under what conditions confidence intervals around the differences in mean costs from a cluster RCT are suitable for estimation using a commonly used cluster-adjusted bootstrap in preference to methods that utilise the Huber-White robust estimator of variance. The bootstrap's main advantage is in dealing with skewed data, which often characterise patient costs. However, it is insufficiently well recognised that one method of adjusting the bootstrap to deal with clustered data is only valid in large samples. In particular, the requirement that the number of clusters randomised should be large would not be satisfied in many cluster RCTs performed to date. METHODS: The performances of confidence intervals for simple differences in mean costs utilising a robust (cluster-adjusted) standard error and from two cluster-adjusted bootstrap procedures were compared in terms of confidence interval coverage in a large number of simulations. Parameters varied included the intracluster correlation coefficient, the sample size and the distributions used to generate the data. RESULTS: The bootstrap's advantage in dealing with skewed data was found to be outweighed by its poor confidence interval coverage when the number of clusters was at the level frequently found in cluster RCTs in practice. Simulations showed that confidence intervals based on robust methods of standard error estimation achieved coverage rates between 93.5% and 94.8% for a 95% nominal level whereas those for the bootstrap ranged between 86.4% and 93.8%. CONCLUSION: In general, 24 clusters per treatment arm is probably the minimum number for which one would even begin to consider the bootstrap in preference to traditional robust methods, for the parameter combinations investigated here. At least this number of clusters and extremely skewed data would be necessary for the bootstrap to be considered in favour of the robust method. There is a need for further investigation of more complex bootstrap procedures if economic data from cluster RCTs are to be analysed appropriately

A Systematic Review Comparing the Acceptability, Validity and Concordance of Discrete Choice Experiments and Best–Worst Scaling for Eliciting Preferences in Healthcare

Objective: The aim of this study was to compare the acceptability, validity and concordance of discrete choice experiment (DCE) and best–worst scaling (BWS) stated preference approaches in health. Methods: A systematic search of EMBASE, Medline, AMED, PubMed, CINAHL, Cochrane Library and EconLit databases was undertaken in October to December 2016 without date restriction. Studies were included if they were published in English, presented empirical data related to the administration or findings of traditional format DCE and object-, profile- or multiprofile-case BWS, and were related to health. Study quality was assessed using the PREFS checklist. Results: Fourteen articles describing 12 studies were included, comparing DCE with profile-case BWS (9 studies), DCE and multiprofile-case BWS (1 study), and profile- and multiprofile-case BWS (2 studies). Although limited and inconsistent, the balance of evidence suggests that preferences derived from DCE and profile-case BWS may not be concordant, regardless of the decision context. Preferences estimated from DCE and multiprofile-case BWS may be concordant (single study). Profile- and multiprofile-case BWS appear more statistically efficient than DCE, but no evidence is available to suggest they have a greater response efficiency. Little evidence suggests superior validity for one format over another. Participant acceptability may favour DCE, which had a lower self-reported task difficulty and was preferred over profile-case BWS in a priority setting but not necessarily in other decision contexts. Conclusion: DCE and profile-case BWS may be of equal validity but give different preference estimates regardless of the health context; thus, they may be measuring different constructs. Therefore, choice between methods is likely to be based on normative considerations related to coherence with theoretical frameworks and on pragmatic considerations related to ease of data collection

Estimating preferences for a dermatology consultation using Best-Worst Scaling: Comparison of various methods of analysis

Background: Additional insights into patient preferences can be gained by supplementing discrete choice experiments with best-worst choice tasks. However, there are no empirical studies illustrating the relative advantages of the various methods of analysis within a random utility framework. Methods: Multinomial and weighted least squares regression models were estimated for a discrete choice experiment. The discrete choice experiment incorporated a best-worst study and was conducted in a UK NHS dermatology context. Waiting time, expertise of doctor, convenience of attending and perceived thoroughness of care were varied across 16 hypothetical appointments. Sample level preferences were estimated for all models and differences between patient subgroups were investigated using ovariateadjusted multinomial logistic regression. Results: A high level of agreement was observed between results from the paired model (which is theoretically consistent with the 'maxdiff' choice model) and the marginal model (which is only an approximation to it). Adjusting for covariates showed that patients who felt particularly affected by their skin condition during the previous week displayed extreme preference for short/no waiting time and were less concerned about other aspects of the appointment. Higher levels of educational attainment were associated with larger differences in utility between the levels of all attributes, although the attributes per use had the same impact upon choices as those with lower levels of attainment. The study also demonstrated the high levels of agreement between summary analyses using weighted least squares and estimates from multinomial models. Conclusion: Robust policy-relevant information on preferences can be obtained from discrete choice experiments incorporating best-worst questions with relatively small sample sizes. The separation of the effects due to attribute impact from the position of levels on the latent utility scale is not possible using traditional discrete choice experiments. This separation is important because health policies to change the levels of attributes in health care may be very different from those aiming to change the attribute impact per se. The good approximation of summary analyses to the multinomial model is a useful finding, because weighted least squares choice totals give better insights into the choice model and promote greater familiarity with the preference data

Rescaling quality of life values from discrete choice experiments for use as QALYs: a cautionary tale

Background: Researchers are increasingly investigating the potential for ordinal tasks such as ranking and discrete choice experiments to estimate QALY health state values. However, the assumptions of random utility theory, which underpin the statistical models used to provide these estimates, have received insufficient attention. In particular, the assumptions made about the decisions between living states and the death state are not satisfied, at least for some people. Estimated values are likely to be incorrectly anchored with respect to death (zero) in such circumstances. Methods: Data from the Investigating Choice Experiments for the preferences of older people CAPability instrument (ICECAP) valuation exercise were analysed. The values (previously anchored to the worst possible state) were rescaled using an ordinal model proposed previously to estimate QALY-like values. Bootstrapping was conducted to vary artificially the proportion of people who conformed to the conventional random utility model underpinning the analyses. Results: Only 26% of respondents conformed unequivocally to the assumptions of conventional random utility theory. At least 14% of respondents unequivocally violated the assumptions. Varying the relative proportions of conforming respondents in sensitivity analyses led to large changes in the estimated QALY values, particularly for lower-valued states. As a result these values could be either positive (considered to be better than death) or negative (considered to be worse than death). Conclusion: Use of a statistical model such as conditional (multinomial) regression to anchor quality of life values from ordinal data to death is inappropriate in the presence of respondents who do not conform to the assumptions of conventional random utility theory. This is clearest when estimating values for that group of respondents observed in valuation samples who refuse to consider any living state to be worse than death: in such circumstances the model cannot be estimated. Only a valuation task requiring respondents to make choices in which both length and quality of life vary can produce estimates that properly reflect the preferences of all respondents