Optimizing Objective Functions Determined from Random Forests

49 Pages Posted: 16 Jun 2017 Last revised: 28 Jan 2018

See all articles by Max Biggs

Max Biggs

Massachusetts Institute of Technology (MIT), Operations Research Center

Rim Hariss

Massachusetts Institute of Technology (MIT), Operations Research Center

Date Written: January 28, 2018

Abstract

We study the problem of optimizing a tree-based ensemble objective with the feasible decisions lie in a polyhedral set. We model this optimization problem as a Mixed Integer Linear Program (MILP). We show this model can be solved to optimality efficiently using Pareto optimal Benders cuts. For large problems, we consider a random forest approximation that consists of only a subset of trees and establish analytically that this gives rise to near optimal solutions by proving analytical guarantees. The error of the approximation decays exponentially as the number of trees increases. Motivated from this result, we propose heuristics that optimize over smaller forests rather than one large one. We present case studies on a property investment problem and a jury selection problem. We show this approach performs well against benchmarks, while providing insights into the sensitivity of the algorithm's performance for different parameters of the random forest.

Keywords: Random Forest, Optimization

JEL Classification: C61

Suggested Citation

Biggs, Max and Hariss, Rim, Optimizing Objective Functions Determined from Random Forests (January 28, 2018). Available at SSRN: https://ssrn.com/abstract=2986630 or http://dx.doi.org/10.2139/ssrn.2986630

Max Biggs (Contact Author)

Massachusetts Institute of Technology (MIT), Operations Research Center ( email )

77 Massachusetts Avenue
Bldg. E 40-149
Cambridge, MA 02139
United States

Rim Hariss

Massachusetts Institute of Technology (MIT), Operations Research Center ( email )

77 Massachusetts Avenue
Bldg. E 40-149
Cambridge, MA 02139
United States

Register to save articles to
your library

Register

Paper statistics

Downloads
437
Abstract Views
1,917
rank
64,239
PlumX Metrics