When Random Assignment is Not Enough: Accounting for Intentional Selectivity in Experimental Research

21 Pages Posted: 6 Oct 2013

See all articles by Yuanping Ying

Yuanping Ying

University of Texas at Dallas

Linda C. Salisbury

Boston College - Carroll School of Management

Fred M. Feinberg

University of Michigan at Ann Arbor - Marketing; University of Michigan, Stephen M. Ross School of Business

Date Written: April 2013

Abstract

A common goal in marketing research is to understand how one evaluates products that have been filtered through some type of screening or selection process. Typical examples include post-choice satisfaction ratings, certain free recall tasks, or the development of consideration sets followed by brand choice. In such situations, behavior is contingent not only on the alternatives being evaluated, but the choice context in which they have become available, creating differing degrees of selectivity. In this paper, we consider situations in which a polytomous selection process limits which items offer up subsequent information. We propose that not flexibly modeling the correlation between choice and evaluation across conditions can lead to systematic, erroneous interpretations of co-variate effects on evaluation. We illustrate this by analyzing two experiments in which subjects choose among, and then evaluate, a frequently-purchased consumer good, as well as data first examined by Ratner, Kahn and Kahneman (1999). Results indicate not only strong selectivity effects, but that traditional specifications of the choice process – presuming the degree of selectivity is invariant across choice contexts – can lead to markedly different interpretations of variable effects. Our findings show that the size and composition of a choice set affects the degree of correlation between choice and evaluation, and further suggest that foregone alternatives can play a disproportionate role in selectivity. Moreover, failing to account for selectivity across experimental conditions, even in well-designed experimental settings, can lead to inaccurate substantive inferences about consumers’ choice processes.

Keywords: Choice Models, Consumer Behavior, Decision-Making, Econometric Models, Sample Selection, Heckman Model, Markov chain Monte Carlo, Hierarchical Bayes

Suggested Citation

Ying, Yuanping and Salisbury, Linda C. and Feinberg, Fred M., When Random Assignment is Not Enough: Accounting for Intentional Selectivity in Experimental Research (April 2013). Ross School of Business Paper No. 1203. Available at SSRN: https://ssrn.com/abstract=2336083 or http://dx.doi.org/10.2139/ssrn.2336083

Yuanping Ying (Contact Author)

University of Texas at Dallas ( email )

2601 North Floyd Road
Richardson, TX 75083
United States

Linda C. Salisbury

Boston College - Carroll School of Management ( email )

140 Commonwealth Avenue
Chestnut Hill, MA 02467
United States

Fred M. Feinberg

University of Michigan at Ann Arbor - Marketing ( email )

Ann Arbor, MI 48109
United States

University of Michigan, Stephen M. Ross School of Business ( email )

701 Tappan Street
Ann Arbor, MI MI 48109
United States

Here is the Coronavirus
related research on SSRN

Paper statistics

Downloads
68
Abstract Views
527
rank
351,555
PlumX Metrics