Rates of Convergence for the Gaussian Mixture Sieve

Chris Genovese and Larry Wasserman

Abstract:

Gaussian mixtures provide a convenient method of density estimation that lies somewhere between parametric models and kernel density estimators. When the number of components of the mixture is allowed to increase as sample size increases, the model is called a mixture sieve. We establish a bound on the rate of convergence in Hellinger distance for density estimation using the Gaussian mixture sieve assuming that the true density is itself a mixture of Gaussians; the underlying mixing measure of the true density is not assumed to necessarily have finite support. Computing the rate involves some delicate calculations since the size of the sieve -- as measured by bracketing entropy -- and the saturation rate, cannot be found using standard methods. When the mixing measure has compact support, using $k_n \sim n^{2/3}/(\log n)^{1/3}$ components in the mixture yields a rate of order $(\log n)^{(1+\eta)/6}/n^{1/6}$ for every $\eta \gt$ .When the mixing measure is not compactly supported, we find that the rates depend heavily on the tail behavior of the true density. The sensitivity to the tail behavior is diminished by using a robust sieve which includes a long-tailed component in the mixture. Then, a spectrum of interesting rates arise depending on the thickness of the tails of the mixing measure.

Keywords: Density Estimation; Mixtures, Rates of Convergence, Sieves

Here is the full postscript text for this technical report. It is 537489 bytes long.