Multi-Surrogate Assisted Multi-Objective Evolutionary Algorithms for Feature Selection in Regression and Classification Problems with Time Series Data

Abstract

Feature selection wrapper methods are powerful mechanisms for reducing the complexity of prediction models while preserving and even improving their precision. Meta-heuristic methods, such as multi-objective evolutionary algorithms, are commonly used as search strategies in feature selection wrapper methods since they allow minimizing the cardinality of the attribute subset and simultaneously maximizing the predictive capacity of the model. However, in high-dimensional problems, multi-objective evolutionary algorithms for wrapper-type feature selection may require excessive computational time, sometimes impractical, especially when the learning algorithm has a high computational cost, such as deep learning. To address this drawback, in this paper we propose a multi-surrogate assisted multi-objective evolutionary algorithm for feature selection, specially designed to improve generalization error. The proposed method has been compared with conventional feature selection wrapper methods that use random forest, support vector machine and long short-term memory learning algorithms to evaluate subsets of attributes. The experiments have been carried out with a regression problem and a classification problem for air quality forecasting with time series data in the south-east of Spain. The results, supported by non-parametric statistical tests, demonstrate the superiority of the proposed multi-surrogate assisted method over conventional wrapper methods using the same run times.

Keywords: Feature selection, multi-objective evolutionary algorithms, surrogate models, regression, classification, time series forecasting