723

**Shrinkage estimators for covariance matrices**

**Michael J. Daniels and Robert E. Kass**

*Revised 05/01*

### Abstract:

Estimation of covariance matrices in small samples has been studied by
many authors. Standard estimators, like the unstructured maximum likelihood
estimator (ML) or restricted maximum likelihood (REML)
estimator can be very
unstable with the smallest estimated eigenvalues being too small and the
largest too big.
A standard approach to more stably estimating the matrix in small samples
is to compute the ML or REML estimator under some
simple structure that involves estimation of fewer parameters,
such as compound symmetry or independence.
However, these estimators will not
be consistent unless the hypothesized structure is correct.
If interest
focuses on estimation of regression coefficients with correlated (or
longitudinal) data, a sandwich estimator
of the covariance matrix may be used to provide standard errors for the
estimated coefficients that are robust in the sense that they remain
consistent under misspecification of the covariance structure.
With large matrices, however, the inefficiency of
the sandwich estimator becomes worrisome.

We consider here two general ``shrinkage'' approaches to estimating the
covariance matrix and regression coefficients.
The first involves shrinking the eigenvalues
of the unstructured ML or REML estimator.
The second involves shrinking an unstructured estimator toward a
structured estimator. For both cases, the data determine the
amount of shrinkage.
These estimators are
consistent and give consistent and asymptotically efficient
estimates for regression coefficients.
Simulations show the improved operating characteristics of the shrinkage
estimators of the covariance matrix
and the regression coefficients in finite samples.
We illustrate our approach on a sleep EEG study which requires
estimation of a 24 x 24 covariance matrix and for
which inferences
on mean parameters critically
depend on the covariance estimator chosen. We
recommend making inference using a particular shrinkage estimator that
provides a reasonable compromise between structured and unstructured
estimators.

*Heidi Sestrich*

*9/5/2000*
Here is the full postscript text for this
technical report. It is 295185 bytes long.