Does the Number of Choice Sets Matter? Results from a Web Survey Applying Discrete Choice Experiment

Abstract

Optimising the design of discrete choice experiments (DCE) is not only a question of how to maximise the a priori parameter efficiency, but should also be concerned with how the nature and complexity of the experiment itself affects model parameters and variance, and hence the outputs such as WTP estimates. The objective of this study was to investigate the impact of the number of DCE choice sets presented to respondents. In order to study behavioural response to the number of choice sets, respondents were given 5, 9 or 17 choice sets in a DCE that elicited preferences for dental services. The results showed no differences between the groups with respect to response rate. No systematic differences were observed in respondents' perception of the uncertainty of their answers. There were some differences in the WTP estimates across the three survey versions, but no differences in standard deviations for WTP estimates or goodness-of-fit statistics. Heteroscedastic conditional logit models revealed somewhat higher variance for the group of respondents who received 17 choice sets compared to those receiving 5 choice sets. This may indicate that respondent cognitive burden increases with the number of choice sets beyond a certain threshold.

Keywords: discrete choice experiments, design properties, design of designs, willingness-to-pay, number of choice sets

JEL Classification: C91, D6, I3