A Big Data Approach to Black Friday Sales

Abstract

Retail companies recognize the need to analyze and predict their sales and customer behavior against their products and product categories. Our study aims to help retail companies create personalized deals and promotions for their customers, even during the COVID-19 pandemic, through a big data framework that allows them to handle massive sales volumes with more efficient models. In this paper, we used Black Friday sales data taken from a dataset on the Kaggle website, which contains nearly 550,000 observations analyzed with 10 features: qualitative and quantitative. The class label is purchases and sales (in U.S. dollars). Because the predictor label is continuous, regression models are suited in this case. Using the Apache Spark big data framework, which uses the MLlib machine learning library, we trained two machine learning models: linear regression and random forest. These machine learning algorithms were used to predict future pricing and sales. We first implemented a linear regression model and a random forest model without using the Spark framework and achieved accuracies of 68% and 74%, respectively. Then, we trained these models on the Spark machine learning big data framework where we achieved an accuracy of 72% for the linear regression model and 81% for the random forest model.

Keywords: Big data; correlation and regression analysis; machine learning; numerical algorithms; performance; prediction; Black Friday sales; cloud