Sequential Model Averaging for High Dimensional Linear Regression Models

Abstract

In high dimensional data analysis, we propose a sequential model averaging (SMA) method to make accurate and stable predictions. Specifically, we in- troduce a hybrid approach that combines a sequential screening process with a model averaging algorithm, where the weight of each model is determined by its Bayesian information (BIC) score (Schwarz, 1978; Chen and Chen, 2008). The sequential technique makes SMA computationally feasible with high dimensional data, because the averaging process assures the prediction’s accuracy and sta- bility. Theoretical results show that SMA not only yields a good model, but also mitigates overfitting. In addition, we demonstrate that SMA provides con- sistent estimators for the regression coefficients and yields reliable predictions under mild conditions. Both simulations and empirical examples are presented to illustrate the usefulness of the proposed method.

Keywords: Forward Regression, Sequential Model Averaging, Sequential Screening, Univariate Model Averaging