Issues in the Use of Ratings-based Versus Choice-based Conjoint Analysis in Operations Management Research

Conjoint analysis has played an important role in helping make a number of operations management decisions including product and service design, supplier selection, and service operations capacity. Many recent advances in this area have raised questions about the most appropriate form of conjoint analysis for this research. We review recent developments in the literature and provide new evidence on how the choice between ratings- and choice-based conjoint models might affect the estimates of customer demand used in operations management models. The biggest systematic difference between ratings-based (RB) and choice-based (CB) parameters is consistent with the compatibility effect, i.e., some enriched attributes like brand name tend to be more important in RB models and some comparable attributes like price are likely to be more important in CB models. Still, there were reasonably small differences between choice- and ratings-based parameters. Parameter similarity was also seen in the lack of differences both in the choice share validations when the ‘‘keep on shopping” alternative was not considered and in the profiles that were predicted to maximize choice shares. This suggests that the two approaches will produce similar estimates of the relative importance of various attributes. In spite of demonstrated success with each method, several reasons lead us to recommend the use of hierarchical Bayesian choice-based conjoint models. First, the slightly higher individual hit rate validations give us greater confidence in predictive accuracy overall as well as an increased ability to target individual customers. Additionally, the greater ease of modeling both changes in market size and competitive reactions are attractive benefits of choice-based models

Customer Choice Modeling in Hospitality Service: A Review of Past Research and Discussion of Some New Applications

Customer choice modeling techniques have grown in sophistication and applicability, so that this methodology can be useful for assessing the services and amenities that are market drivers for customers of hospitality businesses. In essence, customer choice modeling is an experimental process that seeks to identify the key market drivers by comparing respondents’ choices among packages of products and services, known as choice sets. By comparing the ratings of various choice set packages, one can assess which features of a product or service are most critical to the purchase decision. Also helpful in customer choice modeling is analysis of existing purchase patterns, which can be collected from point-of-sale devices and web activity. However, valuable though it is, collected data are backward looking—telling what customers did, but not what they will do. In contrast, customer choice modeling can look forward to see what customers might like, provided the experiment is designed correctly

TCA/HB Compared to CBC/HB for Predicting Choices Among Multi-Attributed Products

For some years, choice-based conjoint analysis (CBC) has demonstrated its superiority over other preference measurement alternatives. So, e.g., in a recent study on German and Polish cola consumers, the superiority of CBC over traditional conjoint analysis (TCA) was striking. As one reason for this superiority, the usage of hierarchical Bayes for CBC parameter estimation was mentioned (CBC/HB). This paper clarifies whether this really makes the difference: Hierarchical Bayes is also used for TCA parameter estimation (TCA/HB). The application to the above mentioned data shows, that this improves the predictive validity compared to TCA but is still inferior to CBC/HB in “high data quality cases". However, in “low data quality cases" TCA/HB is superior to CBC/HB

Comparison of rating-based and choice-based conjoint analysis models. A case study based on preferences for iced coffee in Norway

The authors compare two conjoint analysis approaches eliciting consumer preferences among different product profiles of iced coffees in Norway: rating-based and choice-based conjoint experiments. In the conjoint experiments, stimuli were presented in the form of mock-up pictures of iced coffees varying in coffee type, production origin, calorie content and price, following an orthogonal design. One group of participants (n = 101) performed a rating task of 12 iced coffees whereas another group (n = 102) performed a choice task on 20 iced coffees presented in eight triads. Then, all participants performed self-explicated rating and ranking evaluations of the iced coffee attributes. The rating data were analyzed by a Mixed Model ANOVA while the choice data were analyzed by a Mixed Logit Model. Both models include conjoint factors, demographic variables and their interactions. Results show that the two approaches share similar main results, where consumers prefer low calorie and low price iced coffee products. However, additional effects are detected within each of the two approaches. Further, self-explicated measures indicate that coffee type is the primary attribute for consumers’ selection of iced coffee. The two conjoint approaches are compared and discussed in terms of experimental designs, data analysis methodologies, outcomes, user-friendliness of the results interpretation, estimation power and practical issues

A comparison of two-stage segmentation methods for choice-based conjoint data: a simulation study.

Due to the increasing interest in market segmentation in modern marketing research, several methods for dealing with consumer heterogeneity and for revealing market segments have been described in the literature. In this study, the authors compare eight two-stage segmentation methods that aim to uncover consumer segments by classifying subject-specific indicator values. Four different indicators are used as a segmentation basis. The forces, which are subject-aggregated gradient values of the likelihood function, and the dfbetas, an outlier detection measure, are two indicators that express a subject’s effect on the estimation of the aggregate partworths in the conditional logit model. Although the conditional logit model is generally estimated at the aggregate level, this research obtains individual-level partworth estimates for segmentation purposes. The respondents’ raw choices are the final indicator values. The authors classify the indicators by means of cluster analysis and latent class models. The goal of the study is to compare the segmentation performance of the methods with respect to their success rate, membership recovery and segment mean parameter recovery. With regard to the individual-level estimates, the authors obtain poor segmentation results both with cluster and latent class analysis. The cluster methods based on the forces, the dfbetas and the choices yield good and similar results. Classification of the forces and the dfbetas deteriorates with the use of latent class analysis, whereas latent class modeling of the choices outperforms its cluster counterpart.Two-stage segmentation methods; Choice-based conjoint analysis; Conditional logit model; Market segmentation; Latent class analysis;

Deciphering the plot preferences of forest contractors when purchasing stumpage through conjoint analysis

In order to have sustainable wood value chains, it is essential to understand the factors that determine the incorporation of wood resources into them. Forest contractors are among the key players in this process. This study evaluates the preferences of forest contractors when purchasing forest parcels. The variables considered are: slope of the terrain, distance to the nearest road, parcel size, parcel shape, and the fragmentation of the surrounding land. The study area is Galicia, a region in northwestern Spain that is dominated by small-scale family forestry. An Attribute Levels Survey was designed to establish threshold values of the considered variables to afterwards perform a Choice Based Conjoint Analysis (CBCA). The CBCA allowed to analyze the preferences of forest contractors in relation to the values of these variables and their relative importance. Also, it allowed to generate a map of the level of preference for all the Galician forest parcels with individual, private ownership. The most noteworthy result of this survey was that size greatly impacts the preference of timber contractors, preceded by slope and followed by distance to roads. These results will aid in the design of landscape-scale policies in a geospatial dimension, like the promotion of forest associations, and will lead to an improvement in the sustainability of wood supply.Agencia Estatal de Investigación | Ref. PID2019-111581RB-I00Agencia Estatal de Investigación | Ref. PID2020-118101GB-I00Agencia Estatal de Investigación | Ref. FPU19/02054Universidade de Vigo/CISU

Framing as an App-Design Measure to Nudge Users Toward Infection Disclosure in Contact-Tracing Applications

Contact-tracing applications are only effective in countering current and future pandemics when A) they are widely adopted, and B) users voluntarily disclose their infection to warn others. While much research has investigated how contact-tracing applications should be designed and promoted to motivate app-adoption, little is known about how to increase voluntary infection disclosures. To increase the voluntary infection disclosure among app-users, our joint research project with the core development team of a contact-tracing application, relied on the theory of message framing to investigate how to nudge users toward infection disclosure in contact-tracing applications. Based on a mixed method research design consisting of 15 workshops with the core development team of the contact-tracing application and a conjoint study among 139 users of a European contact-tracing application we show that message framing can be a useful approach to increase the voluntary infection disclosure in contact-tracing applications

Multimodal and nested preference structures in choice-based conjoint analysis: a comparison of bayesian choice models with discrete and continuous representations of heterogeneity

Die Choice-Based Conjoint-Analyse (CBC) ist heutzutage die am weitesten verbreitete Variante der Conjoint-Analyse, einer Klasse von Verfahren zur Messung von Nachfragerpräferenzen. Der Hauptgrund für die zunehmende Dominanz des CBC-Ansatzes in jüngerer Zeit besteht darin, dass hier das tatsächliche Wahlverhalten von Nachfragern sehr realistisch nachgestellt werden kann, indem die Befragten wiederholt ihre bevorzugte Alternative aus einer Menge mehrerer Alternativen (Choice Sets) auswählen. Im Rahmen der CBC-Analyse ist das Multinomiale Logit- (MNL) Modell das am häufigsten verwendete diskrete Wahlmodell. Das MNL-Modell weist jedoch zwei wesentliche Einschränkungen auf: (a) Es impliziert proportionale Substitutionsmuster zwischen den Alternativen, was als Independence of Irrelevant Alternatives- (IIA) Eigenschaft bezeichnet wird, und (b) es berücksichtigt keine Nachfragerheterogenität, da per Definition Teilnutzenwerte für alle Konsumenten als homogen angenommen werden. Seit den 1990er-Jahren werden hierarchisch bayesianische (HB) Modelle für die Teilnutzenwertschätzung in der CBC-Analyse verwendet. Solche HB-Modelle ermöglichen zum einen eine Schätzung individueller Teilnutzenwerte, selbst bei einer beschränkten Datenlage, zum anderen können sie aufgrund der Modellierung von Heterogenität die IIA-Eigenschaft stark abmildern. Der Schwerpunkt der vorliegenden Thesis liegt auf der Verwendung von HB-Modellen mit unterschiedlichen Darstellungen von Nachfragerheterogenität (diskret vs. kontinuierlich) für CBC-Daten sowie außerdem auf einem speziellen HB-Modell, das die IIA-Eigenschaft durch Berücksichtigung von unterschiedlichen Ähnlichkeitsgraden zwischen Teilmengen von Alternativen (Nestern) zusätzlich abschwächt. Insbesondere wird die statistische Performance von einfachen MNL-, Latent Class- (LC) MNL-, HB-MNL-, Mixture-of-Normals- (MoN) MNL-, Dirichlet Process Mixture- (DPM) MNL- und HB-Nested Multinomialen Logit- (NMNL) Modellen (unter experimentell variierenden Bedingungen) hinsichtlich der Recovery von Präferenzstrukturen, der Anpassungsgüte und der Prognosevalidität analysiert. Dazu werden zwei umfangreiche Monte-Carlo-Studien durchgeführt, ferner werden die verschiedenen Modelltypen auf einen empirischen CBC-Datensatz angewandt. In der ersten Monte-Carlo-Studie liegt der Fokus auf dem Vergleich zwischen dem HB-MNL und dem HB-NMNL bei multimodalen und genesteten Präferenzstrukturen. Die Ergebnisse zeigen, dass es keine wesentlichen Unterschiede zwischen beiden Modelltypen hinsichtlich der Anpassungsgüte und insbesondere hinsichtlich der Prognosevalidität gibt. In Bezug auf die Recovery von Präferenzstrukturen schneidet das HB-MNL-Modell zunehmend schlechter ab, wenn die Korrelation in mindestens einem Nest höher ist, während sich das HB-NMNL-Modell erwartungsgemäß an den Grad der Ähnlichkeit zwischen Alternativen anpasst. Die zweite Monte-Carlo-Studie befasst sich mit multimodalen und segmentspezifischen Präferenzstrukturen. Um Unterschiede zwischen den Klassen von Modellen mit unterschiedlichen Darstellungen von Heterogenität herauszuarbeiten, werden hier gezielt die Grade der Heterogenität innerhalb von Segmenten und zwischen Segmenten manipuliert. Unter experimentell variierenden Bedingungen werden die state-of-the-art Ansätze zur Modellierung von Heterogenität (einfaches MNL, LC-MNL, HB-MNL) mit erweiterten Wahlmodellen, die sowohl Nachfragerheterogenität zwischen Segmenten als auch innerhalb von Segmenten abbilden können (MoN-MNL und DPM-MNL), verglichen. Das zentrale Ergebnis dieser Monte-Carlo-Studie ist, dass sich das HB-MNL-Modell, welches eine multivariate Normalverteilung zur Modellierung von Präferenzheterogenität unterstellt, als äußerst robust erweist. Darüber hinaus kristallisiert sich der LC-MNL-Segmentansatz als der beste Ansatz heraus, um die „wahre“ Anzahl von Segmenten zu identifizieren. Abschließend werden die zuvor vorgestellten Wahlmodelle auf einen realen CBC-Datensatz angewandt. Die Ergebnisse zeigen, dass Modelle mit einer kontinuierlichen Darstellung von Heterogenität (HB-MNL, HB-NMNL, MoN-MNL und DPM-MNL) eine bessere Anpassungsgüte und Prognosevalidität aufweisen als Modelle mit einer diskreten Darstellung von Heterogenität (einfaches MNL, LC-MNL). Weiterhin zeigt sich, dass das HB-MNL-Modell für Prognosezwecke sehr gut geeignet ist und im Vergleich zu den anderen (erweiterten) Modellen mindestens ebenso gute, wenn nicht sogar wesentlich bessere Vorhersagen liefert, was für Manager eine zentrale Erkenntnis darstellt.Choice-Based Conjoint (CBC) is nowadays the most widely used variant of conjoint analysis, a class of methods for measuring consumer preferences. The primary reason for the increasing dominance of the CBC approach over the last 35 years is that it closely mimics real choice behavior of consumers by asking respondents repeatedly to choose their preferred alternative from a set of several offered alternatives (choice sets), respectively. Within the framework of CBC analysis, the multinomial logit (MNL) model is the most frequently used discrete choice model. However, the MNL model suffers from two major limitations: (a) it implies proportional substitution rates across alternatives, referred to as the Independence of Irrelevant Alternatives (IIA) property and (b) it does not account for unobserved consumer heterogeneity, as part-worth utilities are assumed to be equal for all respondents by definition. Since the 1990s, Hierarchical Bayesian (HB) models have been used for part-worth utility estimation in CBC analysis. HB models are able to determine part-worth utilities at the individual respondent level even with little individual respondent information on the one hand and, as a result of addressing consumer heterogeneity, can strongly soften the IIA property on the other hand. The focus of the present thesis is on CBC analysis using HB models with different representations of heterogeneity (discrete vs. continuous) as well as using a HB model which mitigates the IIA property to a further extent by allowing for different degrees of similarity between subsets (nests) of alternatives. In particular, we systematically explore the comparative performance of simple MNL, latent class (LC) MNL, HB-MNL, mixture-of-normals (MoN) MNL, Dirichlet Process Mixture (DPM) MNL and HB nested multinomial logit (NMNL) models (under experimentally varying conditions) using statistical criteria for parameter recovery, goodness-of-fit, and predictive accuracy. We conduct two extensive Monte Carlo studies and apply the different types of models to an empirical CBC data set. In the first Monte Carlo study, the focus lies on the comparative performance of the HB-MNL versus the HB-NMNL for multimodal and nested preference structures. Our results show that there seems to be no major differences between both types of models with regard to goodness-of-fit measures and in particular their ability to predict respondents’ choice behavior. Regarding parameter recovery, the HB-MNL model performs increasingly worse when correlation in at least one nest is higher, while the HB-NMNL model adapts to the degree of similarity between alternatives, as expected. The second Monte Carlo study deals with multimodal and segment-specific preference structures. More precisely, to carve out differences between the classes of models with different representations of heterogeneity, we specifically vary the degrees of within-segment and between-segment heterogeneity. We compare state-of-the-art methods to represent heterogeneity (simple MNL, LC-MNL, HB-MNL) and more advanced choice models representing both between-segment and within-segment consumer heterogeneity (MoN-MNL and DPM-MNL) under varying experimental conditions. The core finding from our Monte Carlo study is that the HB-MNL model appears to be highly robust against violations in its assumption of a single multivariate normal distribution of consumer preferences. In addition, the LC-MNL segment solution proves to be the best approach to recover the “true” number of segments. Finally, we apply the previously presented choice models to a real-life CBC data set. The results indicate that models with a continuous representation of heterogeneity (HB-MNL, HB-NMNL, MoN-MNL and DPM-MNL) perform better than models with a discrete representation of heterogeneity (simple MNL, LC-MNL). Further, it turns out that the HB-MNL model works extremely well for predictive purposes and provides at least as good if not considerably better predictions compared to the other (advanced) models, which is an important aspect for managers

When do changes in consumer preferences make forecasts from choice-based conjoint models unreliable?

Forecasting the sales or market share of new products is a major challenge as there is little or no sales history with which to estimate levels and trends. Choice-based conjoint (CBC) is one of the most common approaches used to forecast new products’ sales. However, the accuracy of forecasts based on CBC models may be reduced when consumers’ preferences for the attributes of products are labile. Despite this, there is a lack of research on the extent to which lability can impair accuracy when the coefficients estimated in CBC models are assumed to be constant over time. This paper aims to address this research gap by investigating the prevalence of lability for consumer durable products and its potential impact on the accuracy of forecasts. There are reasons to expect that lability may be particularly evident where a product is subject to rapid technological change and has a short product life-cycle. We carried out a longitudinal survey of the preferences of 161 potential consumers relating to four different types of products. We established that for both functional and innovative products: (i) the CBC models vary significantly over time, indicating changes in consumer preferences and (ii) such changes may cause large differences in forecasts of the probabilities that consumers will purchase particular brands of products. Hence employing models where coefficients do not change over time can potentially lead to inaccurate market share forecasts for high-tech, short life-cycle products that are launched even a short time after the choice-based modelling has been conducted