'Dirty Pool' Revisited: When Less is More

Abstract:     
Among IR scholars, a central empirical proposition is that democracies seek out other democracies as trading partners - the so-called democratic trade hypothesis. Yet, as revealed in a 2001 symposium on Green et al.’s “Dirty Pool” testing this hypothesis is entangled in debates over the appropriate statistical techniques and research design. We use this controversy as a springboard to offer a cautionary tale about how a large data set with a massive N can create overconfidence in hypothesis testing. On the one hand we have over 90,000 dyads of nation-years. On the other hand we can observe only a small number of national transitions in and out of democratic status. These considerations suggest that the proper estimation of a democracy effect (and its standard error) are not readily solved by mechanical resort to statistical formula, particularly with dyads as the units of analysis. Our central contribution is to employ randomization tests on the dyadic analysis to infer the correct p-values associated with the main hypotheses. Second, we model nation-years, where the question is whether the proportion of trade with other democracies increases when a country becomes more democratic. Third, we conduct a difference-in-difference analysis of change in trading partners following democratic or anti-democratic shocks. Finally, we embed our nation-state results in a multi-level framework, distinguishing between the short-term effects of democratic transitions on trade from the long-term effects of national democratic (or not) culture on trade. Rather than adding further layers of statistical complexity, these tests actually are simple and quite intuitive.

Keywords: democracy, trade, permutation tests, hypothesis testing

JEL Classifications: D72, F19, F59, C12, C14

Working Paper Series