A Class of Adaptive Importance Sampling Weighted EM Algorithms for Efficient and Robust Posterior and Predictive Simulation

Tinbergen Institute Discussion Paper No. 12-026/4

37 Pages Posted: 25 Mar 2012

See all articles by Lennart F. Hoogerheide

Lennart F. Hoogerheide

VU University Amsterdam

Anne Opschoor

Vrije Universiteit Amsterdam

H. K. van Dijk

Tinbergen Institute; Econometric Institute

Date Written: March 22, 2012

Abstract

A class of adaptive sampling methods is introduced for efficient posterior and predictive simulation. The proposed methods are robust in the sense that they can handle target distributions that exhibit non-elliptical shapes such as multimodality and skewness. The basic method makes use of sequences of importance weighted Expectation Maximization steps in order to efficiently construct a mixture of Student-t densities that approximates accurately the target distribution - typically a posterior distribution, of which we only require a kernel - in the sense that the Kullback-Leibler divergence between target and mixture is minimized. We label this approach Mixture of t by Importance Sampling and Expectation Maximization (MitISEM). The constructed mixture is used as a candidate density for quick and reliable application of either Importance Sampling (IS) or the Metropolis-Hastings (MH) method. We also introduce three extensions of the basic MitISEM approach. First, we propose a method for applying MitISEM in a sequential manner. Second, we introduce a permutation-augmented MitISEM approach. Third, we propose a partial MitISEM approach, which aims at approximating the joint distribution by estimating a product of marginal and conditional distributions. This division can substantially reduce the dimension of the approximation problem, which facilitates the application of adaptive importance sampling for posterior simulation in more complex models with larger numbers of parameters. Our results indicate that the proposed methods can substantially reduce the computational burden in econometric models like DCC or mixture GARCH models and a mixture instrumental variables model.

Keywords: mixture of Student-t distributions, importance sampling, Kullback-Leibler divergence, Expectation Maximization, Metropolis-Hastings algorithm, predictive likelihood, DCC GARCH, mixture GARCH, instrumental variables

JEL Classification: C11, C22, C26

Suggested Citation

Hoogerheide, Lennart F. and Opschoor, Anne and van Dijk, Herman K., A Class of Adaptive Importance Sampling Weighted EM Algorithms for Efficient and Robust Posterior and Predictive Simulation (March 22, 2012). Tinbergen Institute Discussion Paper No. 12-026/4, Available at SSRN: https://ssrn.com/abstract=2027967 or http://dx.doi.org/10.2139/ssrn.2027967

Lennart F. Hoogerheide (Contact Author)

VU University Amsterdam ( email )

De Boelelaan 1105
Amsterdam, ND North Holland 1081 HV
Netherlands

Anne Opschoor

Vrije Universiteit Amsterdam ( email )

De Boelelaan 1105
Amsterdam, NL 1081 HV
Netherlands

Herman K. Van Dijk

Tinbergen Institute ( email )

Gustav Mahlerplein 117
Burg. Oudlaan 50
Amsterdam/Rotterdam, 1082 MS
Netherlands
+31104088955 (Phone)
+31104089031 (Fax)

HOME PAGE: http://people.few.eur.nl/hkvandijk/

Econometric Institute ( email )

P.O. Box 1738
3000 DR Rotterdam
Netherlands
+31 10 4088955 (Phone)
+31 10 4527746 (Fax)

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
86
Abstract Views
1,219
Rank
527,956
PlumX Metrics