Sequential Model Averaging for High Dimensional Linear Regression Models
36 Pages Posted: 11 Jan 2017 Last revised: 13 Jan 2017
Date Written: January 9, 2017
Abstract
In high dimensional data analysis, we propose a sequential model averaging (SMA) method to make accurate and stable predictions. Specifically, we in- troduce a hybrid approach that combines a sequential screening process with a model averaging algorithm, where the weight of each model is determined by its Bayesian information (BIC) score (Schwarz, 1978; Chen and Chen, 2008). The sequential technique makes SMA computationally feasible with high dimensional data, because the averaging process assures the prediction’s accuracy and sta- bility. Theoretical results show that SMA not only yields a good model, but also mitigates overfitting. In addition, we demonstrate that SMA provides con- sistent estimators for the regression coefficients and yields reliable predictions under mild conditions. Both simulations and empirical examples are presented to illustrate the usefulness of the proposed method.
Keywords: Forward Regression, Sequential Model Averaging, Sequential Screening, Univariate Model Averaging
Suggested Citation: Suggested Citation