Posterior Consistency in Nonparametric Regression Problems under Gaussian Process Priors

Taeryon Choi and Mark J. Schervish


Posterior consistency can be thought of as a theoretical justification of the Bayesian method. One of the most popular approaches to nonparametric Bayesian regression is to put a nonparametric prior distribution on the unknown regression function using Gaussian processes. In this paper, we study posterior consistency in nonparametric regression problems using Gaussian process priors. We use an extension of the theorem of Schwartz (1965) for nonidentically distributed observations, verifying its conditions when using Gaussian process priors for the regression function with normal or double exponential (Laplace) error distributions. We define a metric topology on the space of regression functions and then establish almost sure consistency of the posterior distribution. Our metric topology is weaker than the popular L1 topology. With additional assumptions, we prove almost sure consistency when the regression functions have L1 topologies. When the covariate (predictor) is assumed to be a random variable, we prove almost sure consistency for the joint density function of the response and predictor using the Hellinger metrics.

Keywords:Almost sure consistency, Convergence in probability, Hellinger metric, L1 metric, Laplace distribution

Heidi Sestrich 2004-07-29
Here is the full PDF text for this technical report. It is 324200 bytes long.