Model Selection Procedures and Their Error-Reduction Targets

16 Pages Posted: 24 Mar 2011 Last revised: 10 Apr 2011

See all articles by John R. Doyle

John R. Doyle

Cardiff University - Cardiff Business School

Date Written: April 7, 2011

Abstract

This brief note compares model selection procedures in regression. On the one hand there is an observed error reduction ratio that we calculate from the data: h = SSE2/SSE1, where SSE1 and SSE2 are the sums of squared errors in Models 1 and 2, respectively. On the other hand there is a target ratio, which we call H, which is set by the model selection procedure via an expression involving the numbers of variables and observations. If h < H the procedure accepts Model 2 as superior to Model 1. The aim of this note is to derive expressions for each model selection procedure that have the form H = 1/(1x), where x varies from procedure to procedure. These common-form expressions for H allow us to compare model selection procedures transparently.

The procedures we examine are: Akaike’s Information Criterion (AIC), Bayesian Information Criterion (BIC), Amemiya’s Prediction Criterion (PC), change in R2 (ΔR2), adjusted R2, and adjusted Rf2 which we have created as a simple generalization of adjusted R2. We show that it is less liberal than adjusted R2 and relates closely to AIC and PC: in so doing it provides fresh insight into their properties.

Keywords: Model Selection, AIC, BIC, Prediction Criterion, Adjusted R2, F-Test, Error Reduction Ratio

JEL Classification: C52, C51, C50, C10, C00

Suggested Citation

Doyle, John, Model Selection Procedures and Their Error-Reduction Targets (April 7, 2011). Available at SSRN: https://ssrn.com/abstract=1789907 or http://dx.doi.org/10.2139/ssrn.1789907

John Doyle (Contact Author)

Cardiff University - Cardiff Business School ( email )

Aberconway Building
Colum Drive
Cardiff, CF10 3EU
United Kingdom

HOME PAGE: http://www.cardiff.ac.uk/carbs/faculty/doylejr/index.html

Register to save articles to
your library

Register

Paper statistics

Downloads
161
Abstract Views
860
rank
181,767
PlumX Metrics