Sensitivity Analyses of the Deterrence Hypothesis: Let's Keep the Econ in Econometrics
43 Pages Posted: 2 Nov 1999
Date Written: November 1998
Leamer presents a scathing critique of classical sensitivity analysis concerning regression specification. The argument is that the classical approach lacks a systematic way to determine what explanatory variables truly belong in the regression when there is uncertainty about proper model specification. The remedy offered is Extreme Bound Analysis (EBA). Leamer illustrates the merits of EBA by applying it to a study of the deterrent effect of capital punishment.
EBA aims to determine the sensitivity of estimated effects of one or more "focus" variables of primary interest in a regression equation to the inclusion of additional regressors. It purports to solve the problem by allowing a researcher to partition a set of explanatory variables into "important" and "doubtful" subsets. Only the former is included in the regression with no restrictions. Linear combinations of "doubtful" variables' coefficients, in contrast, are constrained to match the researcher's prior beliefs -- generally zero. EBA produces the largest and smallest estimated coefficients of the focus variables (which themselves could be dubbed either doubtful or important), based on these restrictions. If the computed extreme bounds are too wide, or span zero, the effects of the focus variables are declared too "fragile" to warrant credibility.
Leamer's application of EBA to capital punishment is based on a study of murder rates across US states in 1950 conducted by McManus, and later published by him as a separate paper. Both authors argue that EBA casts serious doubt on the validity of the deterrence hypothesis, at least as it pertains to capital punishment. They conclude that the evidence on deterrence is essentially a product of the researcher's prior beliefs. "Bleeding hearts" could legitimately infer the absence of deterrence, while believers in "eye-for-an-eye" would reject as effective any penalty other than death. "Rational economists" are no more correct in inferring that offenders respond to incentives than are some criminologists who view crime merely as a product of social and environmental ills.
If true, this amounts to an impressive testimony about both the power of EBA and the validity of the economic approach to crime. We suppose that partly for this reason Leamer has titled his paper (supra note 1) grandiloquently "Let's take the con out of econometrics". In this paper we suggest that the best way to keep the con out of econometrics is to keep economics in it. In section 1 we examine the merits of EBA as a replacement for the classical approach. We find the promised remedy worse than the problem it is alleged to cure. The main methodological shortcomings of EBA have already been pointed out by McAleer, Pagan, and Volker [MPV]. Apparently, however, the latter's critique has escaped many economists, perhaps because it has not been backed up by convincing empirical evidence.
In section 2 we use three data sets concerning consumption, production, and human capital investment to show how EBA, as applied by Leamer and McManus [LMC], would find basic economic theory to be fragile. It could dismiss laws of demand, production, and investment as a "con job", however, not because of any faults in the theory, but because of the inherent flaws of EBA itself. These flaws are also shown to be responsible for LMC's premature inferences about the validity of the deterrence hypothesis.
Since LMC's analysis of deterrence is shown to be based on a faulty methodology, we feel an obligation not just to correct the impression their work has left about the strength of the underlying economic model, but also to offer an alternative sensitivity analysis using classical techniques. In section 3 we use similar cross-state data on the incidence of murder as well as other related crimes from both 1940 and 1950 to address a number of potential biases in estimates of supply-of-offenses functions in light of the underlying theory. We focus on 1940 and 1950 not just because LMC used 1950 data, but also because these are the most recent years in which a majority of states in the US still applied capital punishment at considerable annual frequencies.
Our analysis does not necessarily settle the issue. We do show, however, that LMC's analysis does not raise legitimate doubts about empirical results supporting deterrence, let alone prove that these results are just a product of researchers' prior beliefs. More generally, our work indicates that EBA's basic defect is that it does not put any weight on theory as the critical guideline for conducting sensitivity analysis. Indeed, when we conduct a theory-based sensitivity analysis using our cross-state data, we find additional support for the deterrence hypothesis.
JEL Classification: C19
Suggested Citation: Suggested Citation