Putting Big Data Analytics to Work: Feature Selection for Forecasting Electricity Prices Using the Lasso and Random Forests
Journal of Decision Systems, Vol 24, 2015
27 Pages Posted: 5 Aug 2018
Date Written: October 12, 2014
Successful companies are increasingly those companies that excel in the task of extracting knowledge from data. Tapping the source of Big Data requires powerful algorithms combined with a strong understanding of the data used. One of the key challenges in predictive analytics is the identification of relevant factors that may explain the variables of interest. In this paper, we present a case study in predictive analytics in which we focus on the selection of relevant exogenous variables. More specifically, we attempt to predict the EPEX electricity spot prices with reference to historical prices and a deep set of weather variables. In order to choose the relevant weather stations, we use the LASSO and random forests to implicitly execute a variable selection. Overall, in our case study of German weather data we manage to improve forecasting accuracy by up to 16.9% in terms of mean average error.
Keywords: Predictive Analytics\Sep Decision Support; Exogenous Predictors; Feature Selection; Electricity Prices; Weather Data
Suggested Citation: Suggested Citation