Sparse Estimators and the Oracle Property, or the Return of Hodges' Estimator

19 Pages Posted: 7 Apr 2005

See all articles by Hannes Leeb

Hannes Leeb

Yale University - Department of Statistics

Benedikt M. Poetscher

University of Vienna - Statistics

Date Written: April 2007

Abstract

We point out some pitfalls related to the concept of an oracle property as used in Fan and Li (2001, 2002, 2004) which are reminiscent of the well-known pitfalls related to Hodges' estimator. The oracle property is often a consequence of sparsity of an estimator. We show that any estimator satisfying a sparsity property has maximal risk that converges to the supremum of the loss function; in particular, the maximal risk diverges to infinity when ever the loss function is unbounded. For ease of presentation the result is set in the framework of a linear regression model, but generalizes far beyond that setting. In a Monte Carlo study we also assess the extent of the problem infinite samples for the smoothly clipped absolute deviation (SCAD) estimator introduced in Fan and Li (2001). We find that this estimator can perform rather poorly infinite samples and that its worst-case performance relative to maximum likelihood deteriorates with increasing sample size when the estimator is tuned to sparsity.

Keywords: Oracle property, Sparsity, Penalized maximum likelihood, Penalized least squares, Hodges' estimator, SCAD, Lasso, Bridge estimator, Hard thresholding, Maximal risk, Maximal absolute bias, Non-uniform limits

JEL Classification: C51, C20

Suggested Citation

Leeb, Hannes and Poetscher, Benedikt M., Sparse Estimators and the Oracle Property, or the Return of Hodges' Estimator (April 2007). Cowles Foundation Discussion Paper No. 1500, Available at SSRN: https://ssrn.com/abstract=700701

Hannes Leeb (Contact Author)

Yale University - Department of Statistics ( email )

24 Hillhouse Avenue
New Haven, CT 06511
United States
203-432 0639 (Phone)
203-432 0633 (Fax)

Benedikt M. Poetscher

University of Vienna - Statistics ( email )

Universitaetsstr. 5/3
Vienna A-1010
Austria
+43 1 4277 38640 (Phone)