A Review of Statistical Model Selection Criteria: Application to Prediction in Regression, Histograms, and Finite Mixture Models

Abstract

Some model-selection criteria for choosing among a set of alternative models are reviewed. Particular model-selection problems considered here include choice of a regression equation for prediction, the number of bins for a histogram, and the number of component p.d.f.s in a finite mixture model.

Several general methods of scoring the choices in such problems are considered. Minimum description length and penalized likelihood criteria are discussed, in particular AIC (Akaike's Information Criterion), BIC (Bayesian Information Criterion) and KIC (Kashyap's Information Criterion). Interpretation of BIC and KIC in terms of posterior probabilities of alternative models is given. Averaging a prediction or classification over models is considered.

Keywords: model selection, model-selection criteria, subset regression, residual mean square, adjusted R-square, information criteria, AIC, BIC, MDL, minimum description length, histograms, finite mixture model, posterior probabilities of models

JEL Classification: C18