Unintended Causal Pathways: Probing Experimental Mechanisms Through Mediation Analysis
45 Pages Posted: 7 May 2017 Last revised: 10 May 2017
Date Written: May 6, 2017
We present the results of several experimental studies relevant to the general study of causal mechanisms, the specific study of audience costs, and the larger body of survey experiments. First, we show that in the context of audience cost experiments, a portion of the estimated treatment effect is related to the intended causal mechanism, national reputation, but this mechanism underlies a relatively small portion of the total causal effect, leaving much of the causal effect unexplained. Second, we show that the experimental treatment manipulates an unintended mechanism that also underlies part of the treatment effect. We conduct two tests to establish the relative independence of each mediator’s effect. Finally, we investigate the possibility that our findings represent question order effects rather than mediation effects, finding that priming does not substantially threaten our results. Our results and discussion advance our understanding of the challenge of studying causal mediation in survey experiments. We provide some recommendations for survey experimentation moving forward.
Keywords: survey experiments, audience costs, causal mechanisms, mediation analysis, experimental design, public opinion, foreign policy
JEL Classification: F5, F51, C90
Suggested Citation: Suggested Citation