The Design of Stated Choice Experiments: The State of Practice and Future Challenges

Since the work of Louviere and Woodworth (1983) and Louviere and Hensher (1983), stated choice (SC) methods have become the dominant data paradigm in the study of behavioural responses of individuals and households as well as other organizations, in fields as diverse as marketing, transport and environmental and health economics, to name but a few. In SC experiments, it is usual for sampled respondents to be asked to choose from amongst a number of labelled or unlabelled alternatives defined on a number of attribute dimensions, each in turn described by pre-specified levels drawn from some underlying experimental design. The choice task is then repeated a number of times, up to the total number of choice sets being offered over the experiment. Several experimental design strategies are available to the practitioner, however, within the transport literature, it appears that the most common form of experimental design used are orthogonal fractional factorial designs. In this paper we review the properties of such designs, and demonstrate that these properties are unlikely to be retained through to the estimation process. We also discuss an alternative design construction strategy, used to construct statistically optimal designs

Efficiency and Sample Size Requirements for Stated Choice Studies

Stated choice (SC) experiments represent the dominant data paradigm in the study of behavioral responses of individuals, households as well as other organizations, yet little is known about the sample size requirements for models estimated from such data. Current sampling theory does not adequately address the issue and hence researchers have had to resort to simple rules of thumb or ignore the issue and collect samples of arbitrary size, hoping that the sample is sufficiently large enough to produce reliable parameter estimates. In this paper, we demonstrate how to generate efficient designs (based on D-efficiency and a newly proposed sample size S-efficiency measure) using prior parameter values to estimate multinomial logit models containing both generic and alternative-specific parameters. Sample size requirements for such designs in SC studies are investigated. In a numerical case study is shown that a D-efficient and even more an Sefficient design needs a (much) smaller sample size than a random orthogonal design. Furthermore, it is shown that wide level range has a significant positive influence on the efficiency of the design and therefore on the reliability of the parameter estimates

Sample optimality in the design of stated choice experiments

Stated choice (SC) experiments represent the dominant data paradigm in the study of behavioral responses of individuals, households as well as other organizations, yet little is known about the sample size requirements for models estimated from such data. Current sampling theory does not adequately address the issue and hence researchers have had to resort to simple rules of thumb or ignore the issue and collect samples of arbitrary size, hoping that the sample is sufficiently large enough to produce reliable parameter estimates. In this paper, we demonstrate how to generate efficient designs (based on D-efficiency and a newly proposed sample size S-efficiency measure) using prior parameter values to estimate multinomial logit models containing both generic and alternative-specific parameters. Sample size requirements for such designs in SC studies are investigated. Using a numerical case study, we show that using S-efficiency can substantially reduce the sample size required of SC studies

Construction of experimental designs for mixed logit models allowing for correlation across choice observations

In each stated choice (SC) survey, there is an underlying experimental design from which the hypothetical choice situations are determined. These designs are constructed by the analyst, with several different ways of constructing these designs having been proposed in the past. Recently, there has been a move from so-called orthogonal designs to more efficient designs. Efficient designs optimize the design such that the data will lead to more reliable parameter estimates for the model under consideration. The main focus has been on the multinomial logit model, however this model is unable to take the dependency between choice situations into account, while in a stated choice survey usually multiple choice situations are presented to a single respondent. In this paper, we extend the literature by focusing on the panel mixed logit (ML) model with random parameters, which can take the above mentioned dependency into account. In deriving the analytical asymptotic variance-covariance matrix for the panel ML model, used to determine the efficiency of a design, we show that it is far more complex than the crosssectional ML model (assuming independent choice observations). Case studies illustrate that it matters for which model the design is optimized, and that it seems that a panel ML model SC experiment needs less respondents than a cross-sectional ML experiment for the same level of reliability of the parameter estimates

Confidence intervals of willingness-­‐to-­‐pay for random coefficient logit models

Random coefficient logit (RCL) models containing random parameters are increasingly used for modelling travel choices. Willingness-to-pay (WTP) measures, such as the value of travel time savings (VTTS) are, in the case of such RCL models, ratios of random parameters. In this paper we apply the Delta method to compute the confidence intervals of such WTP measures, taking into account the variancecovariance matrix of the estimates of the distributional parameters. Compared to simulation methods such as proposed by Krinsky and Robb, the Delta method is able to avoid some of the simulations by deriving partly analytical expressions for the standard errors. Examples of such computations are shown for different combinations of random distributions

Incorporating model uncertainty into the generation of efficient stated choice experiments: A model averaging approach

Stated choice (SC) studies typically rely on the use of an underlying experimental design to construct the hypothetical choice situations shown to respondents. These designs are constructed by the analyst, with several different ways of constructing these designs having been proposed in the past. Recently, there has been a move from so-called orthogonal designs to more efficient designs. Efficient designs optimize the design such that the data will lead to more reliable parameter estimates for the model under consideration. The literature dealing with the generation of efficient designs has examined and largely solved the issue of a requirement for a prior knowledge of the parameter estimates that will be obtained post data collection. Nevertheless, problems related to the fact that the efficiency of a SC experiment is related to the variance-covariance matrix of the model to be estimated and that different econometric models will have different variance-covariance matrix, thus resulting in different levels of efficiency for the same design, has yet to be addressed. In this paper, we propose the use of a model averaging process over different econometric models to solve this problem. Via the use of a case study, we show that designs generated using the model averaging process prove robust to different model estimation as well as provide decent levels of protection against biased parameter estimates relative to designs generated specifically for a given model type

Approximation of Bayesian Efficiency in Experimental Choice Designs

This paper compares different types of simulated draws over a range of number of draws in generating Bayesian efficient designs for stated choice studies. The paper examines how closely pseudo Monte Carlo, quasi Monte Carlo and polynomial cubature methods are able to replicate the true levels of Bayesian efficiency for SC designs of various dimensions. The authors conclude that the predominantly employed method of using pseudo Monte Carlo draws is unlikely to result in leading to truly Bayesian efficient SC designs. The quasi Monte Carlo methods analyzed here (Halton, Sobol, and Modified Latin Hypercube Sampling) all clearly outperform the pseudo Monte Carlo draws. However, the polynomial cubature method examined in this paper, incremental Gaussian quadrature, outperforms all, and is therefore the recommended approximation method for the calculation of Bayesian efficiency of stated choice designs