Measuring Law’s Normative Force

Abstract

An important question in legal theory and policy is when people are willing to put aside their policy preferences to uphold higher-order legal values. That is, when does constitutional or international law, for instance, have “normative force”? Around two-dozen experimental studies have attempted to measure this question empirically, but their designs contain an inherent limitation. While they are interested in gauging the effect of internalizing a norm, they measure only the effect of exposure to that norm. This is significant because subjects in the treatment group whose priors are strongly contrary to the treatment message on the legality of a policy may effectively be “treatment resistant”: it is difficult to successfully treat them because their prior beliefs on the issue are entrenched; as a result, they simply do not believe the treatment message. This treatment failure attenuates any effects, and where a significant portion of the treatment and/or control group is not successfully treated, the results will be biased toward small, null, or even backfire findings. This article first formally models the mechanism underlying experiments on law’s normative force. I then demonstrate a methodological solution to the problem of treatment resistance. By using the experimental treatment as an instrumental variable and employing a post-treatment treatment-uptake test, the researcher can estimate the causal effect of the real explanatory variable of interest: sincerely holding a belief about a policy’s higher-order lawfulness. Using new data from a 2022 survey experiment conducted on US residents, I illustrate this method for three constitutional or international law issues. The theoretical and empirical results together suggest that backfire effects documented by some studies do not reflect a tepid or negative response against the legal source per se, but rather reflect treatment resistance. These findings suggest that we should re-evaluate the existing body of experimental studies on law’s normative force, and they should prompt researchers to reconsider how we conduct future research in this domain.

Note:

Keywords: Experiments, constitutional law, norms