Putting Big Data Analytics to Work: Feature Selection for Forecasting Electricity Prices Using the Lasso and Random Forests

Journal of Decision Systems, Vol 24, 2015

27 Pages Posted: 5 Aug 2018

See all articles by Nicole Ludwig

Nicole Ludwig

University of Freiburg - Information Systems Research

Stefan Feuerriegel

ETH Zürich

Dirk Neumann

University of Freiburg

Date Written: October 12, 2014

Abstract

Successful companies are increasingly those companies that excel in the task of extracting knowledge from data. Tapping the source of Big Data requires powerful algorithms combined with a strong understanding of the data used. One of the key challenges in predictive analytics is the identification of relevant factors that may explain the variables of interest. In this paper, we present a case study in predictive analytics in which we focus on the selection of relevant exogenous variables. More specifically, we attempt to predict the EPEX electricity spot prices with reference to historical prices and a deep set of weather variables. In order to choose the relevant weather stations, we use the LASSO and random forests to implicitly execute a variable selection. Overall, in our case study of German weather data we manage to improve forecasting accuracy by up to 16.9% in terms of mean average error.

Keywords: Predictive Analytics\Sep Decision Support; Exogenous Predictors; Feature Selection; Electricity Prices; Weather Data

Suggested Citation

Ludwig, Nicole and Feuerriegel, Stefan and Neumann, Dirk, Putting Big Data Analytics to Work: Feature Selection for Forecasting Electricity Prices Using the Lasso and Random Forests (October 12, 2014). Journal of Decision Systems, Vol 24, 2015. Available at SSRN: https://ssrn.com/abstract=3215401

Nicole Ludwig

University of Freiburg - Information Systems Research ( email )

Kollegiengebäude II
Platz der Alten Synagoge
Freiburg, 79098
Germany

Stefan Feuerriegel (Contact Author)

ETH Zürich ( email )

Weinbergstr. 56/58
Zürich, 8092
Switzerland

HOME PAGE: http://www.mis.ethz.ch/

Dirk Neumann

University of Freiburg ( email )

Albert-Ludwigs-Universität Freiburg, Wirtscha.inf.
Kollegiengebäude II, Platz der Alten Synagoge
Freiburg im Breisgau, 79098
Germany

Here is the Coronavirus
related research on SSRN

Paper statistics

Downloads
26
Abstract Views
275
PlumX Metrics