Accuracy Vs. Simplicity: A Complex Trade-Off

51 Pages Posted: 20 Oct 2002

See all articles by Enriqueta Aragones

Enriqueta Aragones

Spanish Council for Scientific Research (CSIC) - Insitute for Economic Analysis

Itzhak Gilboa

Tel Aviv University - Eitan Berglas School of Economics; HEC Paris - Economics & Decision Sciences

Andrew Postlewaite

University of Pennsylvania - Department of Economics

David Schmeidler

Tel Aviv University - Eitan Berglas School of Economics; Ohio State University (OSU)

Date Written: August 2002

Abstract

Inductive learning aims at finding general rules that hold true in a database. Targeted learning seeks rules for the prediction of the value of a variable based on the values of others, as in the case of linear or non-parametric regression analysis. Non-targeted learning finds regularities without a specific prediction goal. We model the product of non-targeted learning as rules that state that a certain phenomenon never happens, or that certain conditions necessitate another. For all types of rules, there is a trade-off between the rule's accuracy and its simplicity. Thus rule selection can be viewed as a choice problem, among pairs of degree of accuracy and degree of complexity. However, one cannot in general tell what is the feasible set in the accuracy complexity space. Formally, we show that finding out whether a point belongs to this set is computationally hard. In particular, in the context of linear regression, finding a small set of variables that obtain a certain value of R2 is computationally hard. Computational complexity may explain why a person is not always aware of rules that, if asked, she would find valid. This, in turn, may explain why one can change other people's minds (opinions, beliefs) without providing new information.

Suggested Citation

Aragonés, Enriqueta and Gilboa, Itzhak and Postlewaite, Andrew and Schmeidler, David, Accuracy Vs. Simplicity: A Complex Trade-Off (August 2002). Available at SSRN: https://ssrn.com/abstract=332382 or http://dx.doi.org/10.2139/ssrn.332382

Enriqueta Aragonés

Spanish Council for Scientific Research (CSIC) - Insitute for Economic Analysis ( email )

08193 Bellaterra
Spain
34-93-580-6612 (Phone)
34-93-580-1452 (Fax)

Itzhak Gilboa

Tel Aviv University - Eitan Berglas School of Economics ( email )

P.O. Box 39040
Ramat Aviv, Tel Aviv, 69978
Israel
972-3-640-6423 (Phone)
972-3-640-9908 (Fax)

HEC Paris - Economics & Decision Sciences

Paris
France

Andrew Postlewaite (Contact Author)

University of Pennsylvania - Department of Economics ( email )

Ronald O. Perelman Center for Political Science
133 South 36th Street
Philadelphia, PA 19104-6297
United States
215-898-7350 (Phone)
215-573-2057 (Fax)

HOME PAGE: http://www.econ.upenn.edu/~apostlew

David Schmeidler

Tel Aviv University - Eitan Berglas School of Economics ( email )

P.O. Box 39040
Ramat Aviv, Tel Aviv, 69978
Israel
+972-3-640-9643 (Phone)
+972-3-640-9357 (Fax)

Ohio State University (OSU) ( email )

Blankenship Hall-2010
901 Woody Hayes Drive
Columbus, OH OH 43210
United States