A Critical Assessment of A Popular Econometric Method for Sensitivity Analysis

Abstract

A popular methodology for sensitivity analysis offers two concrete proposal to draw conclusions about the severity of omitted variable bias: (a) comparing delta* (the value of the sensitivity parameter, delta, consistent with zero treatment effect) with 1, and (b) constructing identified sets under the assumption that delta=1. This methodology suffers from serious problems. First, the key sensitivity parameter used for the robustness analysis, delta, does not measure what it is intended to measure: the relative degree of selection on unobservables. In fact, it cannot be interpreted in a way that is useful for sensitivity analysis. Second, the two concrete proposals for dealing with nonuniqueness of the bias---comparing delta* (the value of delta consistent with zero treatment effect) with 1, and constructing identified sets under the assumption that delta=1---suffer from problems of interpretation and implementation. Finally, simple simulation exercises show that the proposed methodology does not work well even on its own terms.

Keywords: treatment effect, omitted variable bias, proportional selection, sensitivity analysis

JEL Classification: C21