Nonparametric Demand Estimation in the Presence of Unobserved Factors

61 Pages Posted: 5 Dec 2022 Last revised: 27 Mar 2024

See all articles by Sandeep Chitla

Sandeep Chitla

New York University (NYU) - Department of Information, Operations, and Management Sciences

Srikanth Jagabathula

New York University (NYU) - Department of Information, Operations, and Management Sciences

Ashwin Venkataraman

Jindal School of Management, UT Dallas

Date Written: September 21, 2022

Abstract

Customer demand is driven by a host of factors, often in complex ways. Therefore, practical demand models are designed to incorporate a large number of observed features to be able to accurately capture customer preferences. However exhaustive, these models only account for observed features and frequently ignore the impact of unobserved features/factors (UFs). Brand awareness, product shelf positioning, local events, etc., impact demand in complicated ways but are difficult to observe in practice. Unsurprisingly, ignoring the UFs completely results in poor prediction performance.

In this work, we propose a novel method for accounting for UFs when modeling customer demand. We focus on the standard operational setting of a retailer offering its products in multiple stores with differing assortments across stores. The goal of the retailer is to predict demand as a function of the assortment and the prices, for which the retailer has access to historical sales transaction data. We model demand using a mixture of logits model, but do not make any parametric assumptions on the mixing distribution or the underlying mechanism generating the UFs.

Our approach involves treating the UFs as explicit model parameters, resulting in an overparameterized model, and employing a regularization approach which can be flexibly tuned based on the level of noise in the data to control for overfitting. We propose an alternating minimization algorithm to train our model on a large dataset, and establish a sublinear rate of convergence to a stationary point. Using both theoretical and empirical analyses, we show that our method is robust to noise in the data and varying ground-truth mixing distributions. Using real-world grocery sales transaction data, we show that our method scales to real-world sized operational applications and that accounting for product and store-level UFs improves prediction accuracy by more than 63% on standard metrics across 11 product categories over a state-of-the-art benchmark that ignores UFs.

Keywords: nonparametric estimation, omitted variables, mixtures of logit, endogeneity, GMM

Suggested Citation

Chitla, Sandeep and Jagabathula, Srikanth and Venkataraman, Ashwin, Nonparametric Demand Estimation in the Presence of Unobserved Factors (September 21, 2022). Available at SSRN: https://ssrn.com/abstract=4244086 or http://dx.doi.org/10.2139/ssrn.4244086

Sandeep Chitla

New York University (NYU) - Department of Information, Operations, and Management Sciences ( email )

44 West Fourth Street
New York, NY 10012
United States

Srikanth Jagabathula

New York University (NYU) - Department of Information, Operations, and Management Sciences ( email )

44 West Fourth Street
New York, NY 10012
United States

Ashwin Venkataraman (Contact Author)

Jindal School of Management, UT Dallas ( email )

P.O. Box 830688
Richardson, TX 75083-0688
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
198
Abstract Views
907
Rank
283,656
PlumX Metrics