Speeding Up MCMC by Efficient Data Subsampling

Riksbank Research Paper Series No. 121

Sveriges Riksbank Working Paper Series No. 297

39 Pages Posted: 14 Apr 2015

See all articles by Matias Quiroz

Matias Quiroz

Sveriges Riksbank - Research Division; Stockholm University - Department of Statistics

Mattias Villani

Linkoping University

Robert Kohn

University of New South Wales - School of Economics and School of Banking and Finance

Date Written: March 1, 2015

Abstract

The computing time for Markov Chain Monte Carlo (MCMC) algorithms can be prohibitively large for datasets with many observations, especially when the data density for each observation is costly to evaluate. We propose a framework where the likelihood function is estimated from a random subset of the data, resulting in substantially fewer density evaluations. The data subsets are selected using an efficient Probability Proportional-to-Size (PPS) sampling scheme, where the inclusion probability of an observation is proportional to an approximation of its contribution to the log-likelihood function. Three broad classes of approximations are presented. The proposed algorithm is shown to sample from a distribution that is within O(m−½) of the true posterior, where m is the subsample size. Moreover, the constant in the O(m−½) error bound of the likelihood is shown to be small and the approximation error is demonstrated to be negligible even for a small m in our applications.We propose a simple way to adaptively choose the sample size m during the MCMC to optimize sampling efficiency for a fixed computational budget. The method is applied to a bivariate probit model on a data set with half a million observations, and on a Weibull regression model with random effects for discrete-time survival data.

Keywords: Bayesian inference, Markov Chain Monte Carlo, Pseudo-marginal MCMC, Big Data, Probability Proportional-to-Size sampling, Numerical integration

JEL Classification: C11, C13, C15, C55, C83

Suggested Citation

Quiroz, Matias and Villani, Mattias and Kohn, Robert, Speeding Up MCMC by Efficient Data Subsampling (March 1, 2015). Riksbank Research Paper Series No. 121, Sveriges Riksbank Working Paper Series No. 297, Available at SSRN: https://ssrn.com/abstract=2592889 or http://dx.doi.org/10.2139/ssrn.2592889

Matias Quiroz (Contact Author)

Sveriges Riksbank - Research Division ( email )

S-103 37 Stockholm
Sweden

Stockholm University - Department of Statistics ( email )

Stockholm, SE-106 91
Sweden

Mattias Villani

Linkoping University ( email )

Överstegatan 30
Linkoping, 581 83
Sweden

Robert Kohn

University of New South Wales - School of Economics and School of Banking and Finance ( email )

Australian School of Business
Sydney NSW 2052, ACT 2600
Australia
+61 2 9385 2150 (Phone)

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
82
Abstract Views
683
rank
372,288
PlumX Metrics