Optimizing Objective Functions Determined from Random Forests

46 Pages Posted: 16 Jun 2017 Last revised: 29 Jul 2020

See all articles by Max Biggs

Max Biggs

University of Virginia - Darden School of Business

Rim Hariss

McGill University, Desautels Faculty of Management, Students

Georgia Perakis

Massachusetts Institute of Technology (MIT) - Sloan School of Management

Date Written: June 16, 2017

Abstract

We study the problem of optimizing a tree-based ensemble objective with the feasible decisions lie in a polyhedral set. We model this optimization problem as a Mixed Integer Linear Program (MILP). We show this model can be solved to optimality efficiently using Pareto optimal Benders cuts. For large problems, we consider a random forest approximation that consists of only a subset of trees and establish analytically that this gives rise to near optimal solutions by proving analytical guarantees. The error of the approximation decays exponentially as the number of trees increases. Motivated from this result, we propose heuristics that optimize over smaller forests rather than one large one. We present case studies on a property investment problem and a jury selection problem. We show this approach performs well against benchmarks, while providing insights into the sensitivity of the algorithm's performance for different parameters of the random forest.

Keywords: Random Forest, Optimization

JEL Classification: C61

Suggested Citation

Biggs, Max and Hariss, Rim and Perakis, Georgia, Optimizing Objective Functions Determined from Random Forests (June 16, 2017). Available at SSRN: https://ssrn.com/abstract=2986630 or http://dx.doi.org/10.2139/ssrn.2986630

Max Biggs (Contact Author)

University of Virginia - Darden School of Business ( email )

P.O. Box 6550
Charlottesville, VA 22906-6550
United States

Rim Hariss

McGill University, Desautels Faculty of Management, Students ( email )

1001 Sherbrooke St. West
Montreal
Canada

Georgia Perakis

Massachusetts Institute of Technology (MIT) - Sloan School of Management ( email )

100 Main Street
E62-565
Cambridge, MA 02142
United States

Here is the Coronavirus
related research on SSRN

Paper statistics

Downloads
719
Abstract Views
3,079
rank
38,772
PlumX Metrics