A Simple Theory of Scientific Learning

Abstract:     
Scientists use diverse evidence to learn about the relative validity of various broad theories. Given the lack of statistical structure in this scientific learning problem, techniques of model selection and meta-analysis are not directly useful as quantitative guides. I use five simplifying assumptions to make the problem tractable by standard statistical methods. Combining Bayesian and frequentist approaches, I derive simple, intuitive rules for updating beliefs. The theory incorporates trade-offs among seemingly incomparable dimensions often used to judge models: ex-ante plausibility, precision, empirical accuracy and general applicability. I establish necessary and sufficient conditions for the consistency of the learning procedure which provides easy robustness checks for applied analysis and a simple algorithm for choosing a robustly consistent, but efficient, trade-off between precision and accuracy. I develop the theory in the context of data collected by Charness and Rabin (2002). In contrast to the authors' analysis, I find (for a wide range of prior beliefs and parameter values) that after taking into account its greater precision, Selfishness is the best model of choice in the simple games they consider.

Number of Pages in PDF File: 45

Keywords: model selection, machine learning, other-regarding preferences, Bayesian statistics

JEL Classification: B41. C11, C52

working papers series