Modeling Covariates with Nonparametric Bayesian Methods

Abstract:     
A research problem that has received increased attention in recent years is extending Bayesian nonparametric methods to include dependence on covariates. Limited attention, however, has been directed to the following two aspects. First, analyzing how the performance of such extensions differs, and second, understanding which features are worthwhile in order to produce better results. This article proposes answers to those questions focusing on predictive inference and continuous covariates. Specifically, we show that 1) nonparametric models using different strategies for modeling continuous covariates can show noteworthy differences when they are being used for prediction, even though they produce otherwise similar posterior inference results, and 2) when the predictive density is a mixture, it is convenient to make the weights depend on the covariates in order to produce sensible estimators. Such claims are supported by comparing the Linear DDP (an extension of the Sethuraman representation) and the Conditional DP (which augments the nonparametric distribution to include the covariates). Unlike the Conditional DP, the weights in the predictive mixture density of the Linear DDP are not covariate-dependent. This results in poor estimators of the predictive density. Specifically, in a simulation example, the Linear DDP wrongly introduces an additional mode into the predictive density, while in an application to a pharmacokinetic study, it produces unrealistic concentration-time curves.

Number of Pages in PDF File: 28

Keywords: Dirichlet process mixture, Hierarchical model, Nonparametric Bayes, Covariates modeling, Dependent Dirichlet process

working papers series