Boosting Rvfl Networks: An Outlier-Resistant Learning Framework

Abstract

The presence of noise and outliers in real-world data significantly affects the performance of classification and regression models. A common strategy to mitigate the impact of outliers is to apply fuzzy concepts or weighting functions; however, these approaches are highly dependent on the selection of fuzzy functions. In this paper, we present a powerful approach that merges the adaptive strength of AdaBoost with the flexibility of random vector functional link (RVFL) networks, creating an effective solution for tackling outliers and enhancing model robustness. While AdaBoost is typically effective, its exponential loss function can escalate rapidly in the presence of outliers, resulting in suboptimal performance. Consequently, combining RVFL with classical AdaBoost does not adequately reduce the influence of outliers, thereby compromising the performance of RVFL, which is inherently a strong classifier. To address this limitation, we propose a novel model that replaces AdaBoost's traditional exponential loss function with a bounded exponential loss, leading to the development of the AdaBoost-based RVFL network with bounded loss (AD-RVFL). Additionally, we extend this model to regression tasks, introducing AD-RVFL-R. To further assess the effects of the exponential loss in AdaBoost, we introduce a comparison model using the classical exponential loss function, termed classical AD-RVFL (CAD-RVFL). We conducted extensive experiments on real-world datasets, which demonstrated that the proposed AD-RVFL and AD-RVFL-R models outperform their baseline counterparts. Moreover, experiments with varying outlier ratios confirmed that AD-RVFL exhibits strong resistance to outliers, making it a robust and reliable solution for both classification and regression tasks.

Keywords: Random Vector Functional Link (RVFL) Network, Adaboost, Bounded Exponential Loss, Ensemble.