Re-Em Trees: A New Data Mining Approach for Longitudinal Data

35 Pages Posted: 15 Jul 2009

See all articles by Rebecca J. Sela

Rebecca J. Sela

New York University (NYU) - Leonard N. Stern School of Business; J.P. Morgan Chase & Co.

Jeffrey S. Simonoff

New York University (NYU) - Leonard N. Stern School of Business; New York University (NYU) - Department of Information, Operations, and Management Sciences

Date Written: June 2009

Abstract

Longitudinal data refer to the situation where repeated observations are available for each sampled individual. Methodologies that take this structure into account allow for systematic differences between individuals that are not related to covariates. A standard methodology in the statistics literature for this type of data is the random effects model, where these differences between individuals are represented by so-called “effects” that are estimated from the data. This paper presents a methodology that combines the flexibility of tree-based estimation methods with the structure of random effects models for longitudinal data. We apply the resulting estimation method, called the RE-EM tree, to pricing in online transactions, showing that the RE-EM tree is less sensitive to parametric assumptions and provides improved predictive power compared to linear models with random effects and regression trees without random effects. We also perform extensive simulation experiments to show that the estimator improves predictive performance relative to regression trees without random effects and is comparable or superior to using linear models with random effects in more general situations.

Suggested Citation

Sela, Rebecca J. and Simonoff, Jeffrey S., Re-Em Trees: A New Data Mining Approach for Longitudinal Data (June 2009). NYU Working Paper No. 2451/28094. Available at SSRN: https://ssrn.com/abstract=1433938

Rebecca J. Sela (Contact Author)

New York University (NYU) - Leonard N. Stern School of Business ( email )

44 West 4th Street
Suite 9-160
New York, NY NY 10012
United States

J.P. Morgan Chase & Co. ( email )

60 Wall St.
New York, NY 10260
United States

Jeffrey S. Simonoff

New York University (NYU) - Leonard N. Stern School of Business ( email )

44 West 4th Street
Suite 9-160
New York, NY NY 10012
United States

New York University (NYU) - Department of Information, Operations, and Management Sciences

44 West Fourth Street
New York, NY 10012
United States

Here is the Coronavirus
related research on SSRN

Paper statistics

Downloads
171
Abstract Views
917
rank
181,637
PlumX Metrics