Learning From Biased Research Designs

Abstract

Most contemporary empirical work in political science aims to learn about causal effects from research designs that may be subject to bias. We provide a Bayesian framework for understanding how researchers should approach the general problem of inferring causal effects from potentially biased research designs. The key to our approach is that both researchers and their audiences have prior beliefs about both causal effects and the degree and direction of bias. Once these priors are specified, what a rational researcher should learn from a potentially biased estimate can be derived from Bayes’ rule. We apply this principle to explore when we should learn more or less from basic difference of means estimates, and then extend our analysis to speak to common modern designs intended to uncover causal effects.

Keywords: Research Design, Causality, Treatment Effects, Bayes, Learning, Identification, Bias