Sequential Monte Carlo for Hierarchical Bayes with Large Datasets

Abstract

Practical use of Hierarchical Bayes models require the availability of efficient methods for posterior inference. Sequential Monte Carlo methods have appeared as an extremely robust way to simulate complicated Bayesian posteriors. The simplest version successively weights and resample draws called particles from a sequence of target distributions. The method has a main weakness keeping it from being used in complex hierarchical models and large data sets: sample impoverishment. This issue is usually alleviated through the use of a "refreshing" MCMC step. However, to preserve effectiveness, the computational costs are quadratically increasing with the total number of observations. We propose a new SMC-within-SMC method. In a first step, each individual-level parameter is estimated separately using standard SMC and a non-hierarchical auxiliary prior. In a second step, we use weighting methods to replace the auxiliary prior with the hierarchical one without the need to recompute any likelihood. In addition to allowing for the separate processing of individual data, this approach drastically reduces the computational costs. A MATLAB package is provided.

Keywords: Sequential Monte Carlo, Sample Degeneracy, Sample Impoverishment, Hierarchical Bayes, Big Data