New Global Optimization Algorithms for Model-Based Clustering

37 Pages Posted: 18 Jul 2009

See all articles by Jeffrey Heath

Jeffrey Heath

Centre College

Michael Fu

University of Maryland - College Park

Wolfgang Jank

University of Maryland - Decision and Information Technologies Department

Date Written: July 15, 2009

Abstract

The Expectation-Maximization (EM) algorithm is a very popular optimization tool for mixture problems and in particular for model-based clustering problems. However, while the algorithm is convenient to implement and numerically very stable, it only produces local solutions. Thus, it may not achieve the globally optimal solution in problems that have a large number of local optima. This paper introduces several new algorithms designed to produce global solutions in model-based clustering. The building blocks for these algorithms are methods from the operations research literature, namely the Cross-Entropy (CE) method and Model Reference Adaptive Search (MRAS). One problem with applying these methods directly is the efficient simulation of positive definite covariance matrices. We propose several new solutions to this problem. One solution is to apply the principles of Expectation-Maximization updating, which leads to two new algorithms, CE-EM and MRAS-EM. We also propose two additional algorithms, CE-CD and MRAS-CD, which rely on the Cholesky decomposition. We conduct numerical experiments of varying complexity to evaluate the effectiveness of the proposed algorithms in comparison to classical EM. We find that although a single run of the new algorithms is slower than a single run of EM, all have the potential for producing significantly better solutions. We also find that although repeat application of EM may achieve similar results, our algorithms provide automated, data-driven decision rules which may significantly reduce the burden of searching for the global optimum.

Keywords: EM algorihm, global optimum, mixture model

JEL Classification: C61

Suggested Citation

Heath, Jeffrey and Fu, Michael and Jank, Wolfgang, New Global Optimization Algorithms for Model-Based Clustering (July 15, 2009). Available at SSRN: https://ssrn.com/abstract=1434390 or http://dx.doi.org/10.2139/ssrn.1434390

Jeffrey Heath

Centre College ( email )

600 West Walnut Street
Danville, KY 40422
United States

Michael Fu

University of Maryland - College Park ( email )

College Park, MD 20742
United States

Wolfgang Jank (Contact Author)

University of Maryland - Decision and Information Technologies Department ( email )

Robert H. Smith School of Business
4300 Van Munching Hall
College Park, MD 20742
United States
301-405-1118 (Phone)

HOME PAGE: http://www.smith.umd.edu/faculty/wjank/

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
66
Abstract Views
1,012
Rank
612,800
PlumX Metrics