\documentstyle[12pt,epsfig,fullpage,fancyheadings]{article}
\pagestyle{fancy}
\lhead{Statistics 36-303}
\rhead{Spring, 2003}
\setlength{\headrulewidth}{0pt}

\begin{document}
\title{Notes on Cluster Sampling\\
for Statistics 36-303: Sampling, Surveys and Society}

\author{Stephen E. Fienberg\\
Department of Statistics\\
 Carnegie Mellon University} 
\maketitle

\section*{I. Why Use Clustering?}
        After stratification the most natural extension to simple
        random sampling involves the use of clusters of the population
        of interest.  In stratified sampling, we divide the population
        into distinct subpopulations called {\em strata}, and within each
        stratum we select a separate sample.  In cluster sampling, we
        divide the population up into clusters, and we select a sample
        of clusters and include all of the elements from these
        clusters in the sample.  Figure 1, reproduced from Lohr
        (1999), illustrates diagrammatically the similarities and
        differences between these approaches.

%\begin{center}
%\epsfig{file=lohr.eps}
%\end{center}

 There are two primary reasons for clustering:

\begin{enumerate}
\item It is often the case that a reliable list of elements of the population is unavailable and it is unreasonably expensive to try to compile such a list.  We can, however, make a list of clusters and thus it is sensible to use them as the sampling units.  This is often the case when we sample human populations and the clusters are households.  This is because it is relatively easy to prepare and maintain a list of household locations, whereas it is virtually impossible to maintain a list of individuals in identifiable locations.
\item  Even when a list of individual elements of the population is available, the cost incurred in actually locating one element of a cluster is far greater than that associated with getting the information from all other elements.  For example the travel costs associated in going from one housing unit to another in a random sample of units may be substantial.  Further, when the cluster consists of a household, one individual can provide information on all the other members. 
\end{enumerate}     


\newpage
\mbox{Figure 1. Similarities and Differences Between Cluster Sampling and
  Stratified Sampling}\\
\newpage


   Unlike stratification, which if used wisely increases the precision of our sample for a fixed overall sample size, clustering tends to decrease the precision of our sample for a fixed overall sample size.  This is because individuals within a cluster tend to be more alike with respect to various characteristics than those between clusters.  The basic ideas about clustering are relatively easy to understand and in these notes we explore the effects of clustering primarily by example, concentrating on the estimation of attributes.  The basic ideas carry over to estimation using measurements and we indicate the nature of the impact of clustering in such situations.  Later in the course, we will examine in detail the design actual surveys that combine both clustering and stratification to achieve the advantages in cost from the former while preserving some of the precision that stratification confers.


We limit our attention in these notes to clusters of equal size, primarily to keep the notation and the algebra simple, but the basic ideas of cluster sampling carry over to unequal size clusters.

\section*{II. Some Basic Elements of Cluster Sampling} 
        

{\bf Example 1:} Suppose we have a population of size $2N$ composed of $N$ families of size 2, a husband and wife.  We say that the families are clusters of size $M=2$.  Further suppose that the husband and wife in any given pair are exactly the same age. We are interested in the proportion of the population eligible for Medicare corresponding to those over the age of 65, and we take a sample of n families.  Since both members of each family have the same age we in effect have redundant information and instead of ending up with an overall sample of size $2n$ individuals, our effective sample size is only $n$.
      

Now suppose that husbands and wives don't have identical ages, but on average older husbands have older wives and younger husbands have younger wives.  This positive association or correlation between the age of the husband and the age of a wife in a pair again reduces the ``effective sample size'' associated with our cluster sample of $n$ families.  We get an estimate that is more accurate than a simple random sample of n individuals from the population, but still less than a simple random sample of size $2n$.
        

This example illustrates the basic impact of clustering that we will tend to observe in the sampling of human populations.  In general we consider a population of $NM$ elements subdivided into $N$ clusters of size $M$.  We take a sample of $n$ of these clusters and incorporate into our sample information on all $M$ elements in each of the selected clusters.  Thus we record information on $nM$ units.  If the information from individuals within a cluster is positively related, then there is less variation among individuals within a cluster than for the same number of individuals drawn from different clusters.  Thus we expect that our cluster sample will be less accurate than a simple random sample of the same size, $nM$.
       

The actual formulas for the variance of an estimate for a proportion or for a mean from a cluster sample involve measures of variation within and between clusters.

\section*{III. Cluster Sampling for Attributes}
        We now consider the estimation of the proportion $p$ of the population of size $NM$ possessing an attribute.  Let $p_i$ be the proportion of elements in the $i$th cluster possessing the attribute.  Then the cluster sample estimate of $p$, $\overline{p}_c$, is just the average of the values of $p_i$ for the sampled clusters,
        
\begin{eqnarray}
\overline{p}_c=\frac{1}{n}\sum^n_{i=1}p_i
\end{eqnarray}


\noindent and, since the clusters are of equal size, this estimate is simply the
        overall sample proportion of individuals with the attribute of
        interest.  Thus our cluster sample selection procedure assigns
        each individual in the sample the same chance of selection and
        the sample is {\em self-weighting.} 
      

We get the variance of $\overline{p}_c$ by treating the values of $p_i$ as a sample of $n$ measurements and looking at their variation when used to estimate the overall population proportion $p$, i.e.,

\begin{eqnarray}
Var(\overline{p}_c)=\frac{(N-n)}{(N-1)}\frac{\Bigg(\frac{\sum^N_{i=1}(p_i-p)^2}{N}\Bigg)}{n}.      
\end{eqnarray}


We typically want to compare the accuracy of our estimate, $\overline{p}_c$, with that of a simple random sample of the same overall sample size, $nM$, i.e., 
\begin{eqnarray}
Var(\overline{p})=\frac{(NM-nM)}{(NM-1)}\frac{p(1-p)}{nM}.
\end{eqnarray}         


One way to compare the variances involves the ``correlation'' between
        elements in the same cluster, $\rho$.  This quantity $\rho$ is
        called the {\em intracluster correlation coefficient.}  It
        turns out that\\
  
%box around
\hspace*{1in}\fbox{\parbox[c][.5in]{4in}{
\begin{eqnarray*}
\frac{Var(\overline{p}_c)}{Var(\overline{p})}=1+(M-1)\rho.
\end{eqnarray*}}}\hfill
\parbox{1in}{\begin{eqnarray}\end{eqnarray}}\\       


\noindent Since variances cannot be negative, the quantity $1+(M-1)\rho$ can't be negative and the minimum possible value for $\rho$ is $\frac{-1}{(M-1)}$, which tends to 0 as the cluster size $M$ gets large.  Unlike a regular correlation coefficient which takes values between 1 and -1, the intracluster correlation coefficient runs between 1 and $\frac{-1}{(M-1)}$ .  If $\rho>0$, the cluster provides less precision than a random sample of $M$ individuals, whereas when $\rho<0$, something which occasionally happens, the use of clusters is more precise.
      
\vspace{.25in}
\noindent {\bf Example 2: Limited English Proficiency Students.}  The
U. S. Department of Education is interested in determining the number
of elementary school children in public schools with limited
proficiency in English (LEP).  Suppose there are $N=20$ schools in a
given district and that each school has $M=100$ students.  Investigators take a sample of $n=5$ schools and gather information on the proportion of LEP students in each school. The total sample size is number of schools sampled times the number of children in each school, i.e., 500.  Because of housing patterns we expect children in a given school to be more alike with respect to their proficiency in English than those in different schools. This is because immigrants to the district from a given country often live close to one another for economic or other reasons.  Suppose we can determine that the intra-school correlation coefficient for the attribute LEP is 0.0383.  Then the ratio of the variance of the cluster sample of 5 clusters to the variance we would have had if we had taken a simple random sample of students from a across the school district of size 500 is

$$
1+(M-1)\rho=1+99 \times0.0383=4.79        
$$


\noindent i.e., the variance of the cluster sample is almost 5 times greater than that of a simple random sample of equivalent size.  Put another way, we could have taken a simple random sample of 100 children from across the school district and achieved an estimate of the proportion of LEP students with equivalent accuracy.


Why then did investigators choose their sample in this way?  The
answer is cost.  Suppose that the cost of going to a school and
setting up a language test is \$1000, whereas the cost of
administering the test to the student one set up is \$10.  Taking a
random sample of 100 students would have meant that they would have
gone to at least, say, 10 schools, perhaps more.  The cost of
administering the test in 10 schools is

$$
(10 \times\$1000)+(100 \times\$10)=\$11,000.
$$
         
\noindent Instead, the investigators went to only 5 schools and thus their cost was


$$
(5 \times\$1000)+(100 \times\$10)=\$6,000.
$$        


\noindent Thus, by taking a cluster sample the investigators incurred only (6/11)th of the cost associated with a simple random sample of equivalent precision.


When the population size is large and we can ignore the finite population correction, and we can rewrite the ratio of the variance for cluster sample versus that for a simple random sample directly as


\begin{eqnarray}
\frac{Var(\overline{p}_c)}{Var(\overline{p})} \cong
\frac{M\sum^N_{i=1}(p_i-p)^2}{Np(1-p)}
\end{eqnarray}
        

What we have in these formulas for the variances is three different measures of variability: (i) the overall population variance, i.e., the variance of the observations for samples of size 1, $p(1-p)$;  (ii) the subpopulation for the $i$th cluster, i.e., variance of the observations for samples of size 1 within the $i$th cluster, $p_i(1-p_i)$; and  (iii) the variation of the cluster proportions about the overall population proportion, $\sum(p_i-p)^2$.
        

A very famous formula in statistics links these quantities as follows:\\

\hspace*{1in}\fbox{\parbox[c][.7in]{4in}{
\begin{eqnarray*}
NMp(1-p)=M\sum^N_{i=1}(p_i-p)^2+M\sum^n_{i=1}p_i(1-p_i)\\
\mbox{total SS=SS between, clusters+SS within clusters}\nonumber
\end{eqnarray*}}}\hfill
\parbox{1in}{\begin{eqnarray}\end{eqnarray}}\\

        
By substituting for the sum of squares between clusters the formula
for the ratio of variances we get

\begin{eqnarray}
\frac{Var(\overline{p}_c)}{Var(\overline{p})}
& \cong &\frac{M\sum^N_{i=1}(p_i-p)^2}{Np(1-p)}\\
 & = &\frac{NMp(1-p)-M\sum^N_{i=1}p_i(1-p_i)}{Np(1-p)}\nonumber\\
& = &M\Bigg(1-\frac{\sum^N_{i=1}p_i(1-p_i)}{Np(1-p)}\Bigg)\nonumber
\end{eqnarray}\label{num7}
        
The fraction in expression (7) in brackets tends is always less than or equal to 1.  Thus the variance of cluster sampling is never more than M times that of an equivalently sized simple random sample. But often the fraction is non-negligible and we get little degradation from the clustering relative to simple random sampling.  In the other extreme, when all of the clusters are identical and $p_i=p$ for all clusters, the fraction is 1 and ratio of the variances is approximately equal to 0, i.e., cluster sampling has yielded a fantastic gain: in this instance, once you've seen one cluster you in effect have seen them all!    Williams (1978, Chapter 11) gives a simple example of this phenomenon.
       
\vspace{.25in}
\noindent {\bf Example 3: Numerical Illustrations.}  Suppose that the proportion
of individuals in a population with a specific attribute is $p=0.5$.
If the population consists of $N=20$ clusters, where for 10 clusters
$p_i=0.25$ and for the other 10 $p_i=0.75$, then        


\begin{eqnarray}
\frac{Var(\overline{p}_c)}{Var(\overline{p})} & \cong &
M\Bigg( 1-\frac{\sum^N_{i=1}p_i(1-p_i)}{Np(1-p)} \Bigg)\\
& = & M\Bigg( 1-\frac{10(0.25 \times0.75)+10(0.75 \times0.25)}{20(0.5
\times0.5)} \Bigg)\nonumber\\ 
 & = & M \bigg(1-\frac{3}{4} \bigg)= \frac{M}{4}. \nonumber
\end{eqnarray}


Thus for clusters of size $M=2$ and $M=3$, there is less variability
        in a cluster sample than in a simple random sample!  For $M
        \geq4$, however, cluster sampling is less efficient, since the ratio is less than 1. As the cluster size $M$ increases in this example, we have greater and greater degradation in precision associated with cluster sampling. 
       
\vspace{.25in}
\noindent {\bf Example 2 (Cont.):}  Suppose the distribution of the number of
LEP students in the 20 schools in our school district is as follows:

\vspace{.25in}
\begin{tabular}{c|c}
    {\bf Number of School} & {\bf Proportion of LEP Students}\\ \hline
2 & 0.1\\
3 & 0.2\\
3 & 0.3\\
10 & 0.4\\
2 & 0.5\\
\end{tabular}

\vspace{.25in}
The overall proportion of LEP students is $p=0.326$.  Thus $Np(1-p)=20
\times0.326 \times0.674=4.39$ and from the information in the table we calculate
                
\begin{eqnarray}
\sum^N_{i=1}p_i(1-p_i)=4.19.
\end{eqnarray}


Since $M=100$ the ratio of the variances is

\begin{eqnarray}
\frac{Var(\overline{p}_c)}{Var(\overline{p})} \cong
        M\Bigg(1-\frac{\sum^N_{i=1}p_i(1-p_i)}{Np(1-p)}
        \Bigg)=100\Bigg(1-\frac{4.19}{4.39}\Bigg)=4.56.
\end{eqnarray} 
        

Thus we get the precision of the cluster sample is equivalent to that of a simple random sample of about 20\% the sample size.  The difference between the ratio of 4.56 computed here and that of 4.79 computed in our earlier look at this example is due to ignoring the finite population correction here.

\section*{IV. Cluster Sampling for Measurements}
        The same ideas carry over for cluster samples of measurements.
        We now consider the estimation of the mean $\mu$ of the
        population of size $NM$.  Let $y_i$ be the sum of measurements
        for the $i$th cluster.  Then the cluster sample estimate of
        $\mu$, $\overline{y}_c$, is just the average of the all of
        measurements for all n sampled clusters

\begin{eqnarray}
\overline{y}_c=\frac{1}{nM}\sum^n_{i=1}y_i.
\end{eqnarray}


Our cluster sample selection procedure assigns each individual in the
        sample the same chance of selection and the sample is {\em
        self-weighting}.  We get the variance of $\overline{y}_c$ by treating the values of $y_i$ as a sample of $n$ measurements and looking at their variation about their mean value for all $N$ clusters.  As we did with attributes, we compare the accuracy of our estimate, $\overline{y}_c$, with that of a simple random sample $\overline{y}$ of the same overall sample size, $nM$, and the ratio turns out to be the same as it was for attributes:
        

\begin{eqnarray}
\frac{Var(\overline{y}_c)}{Var(\overline{y})}=1+(M-1)\rho,
\end{eqnarray}


\noindent where $\rho$ is the intracluster correlation coefficient.
        
\vspace{.25in}
\noindent {\bf Example 4:} Henry (1990, pp. 107-109) gives an example where   and , in which the estimated variances are 
 
\begin{eqnarray}
s^2(\overline{y}_c)=30.17\nonumber\\
s^2(\overline{y})=20.23\nonumber
\end{eqnarray}


Thus there is an increase in the variance of about 50\% due to clustering. This means that a simple random sample of equivalent precision to the cluster sample would have required only a sample of size

$$
       \bigg(\frac{20.23}{30.17}\bigg)15 = 10.05
$$

or 10.  For this example, the estimated intracluster correlation coefficient is
         
\begin{eqnarray}
\overline{\rho}=\frac{1}{M-1}\bigg(\frac{s^2(\overline{y}_c)}{s^2(\overline{y})}-1\bigg)=\frac{1}{4}\bigg(\frac{30.17}{20.23}-1\bigg)=0.123.
\end{eqnarray}


\section*{V. Unequal Cluster Sizes}
        The results for clustering described here work out in a similar way when the clusters are not of equal size although the actual formulas are somewhat more complicated.  Choosing compact clusters of roughly equal size turns out to be the most efficient way to do cluster sampling, but in real life clusters often come in varying sizes and there is little we can do about this. For example, households come in sizes that typically vary from 1 (for single person households) to as many as 12, or even more.  The ratio of the variance for a cluster sample to that of a simple random sample still takes approximately the same form,

\begin{eqnarray}
\frac{Var(\overline{y}_c)}{Var(\overline{y})}
\cong1+(\overline{M}-1)\rho,
\end{eqnarray}
        

where $\overline{M}$ is the average cluster size.

\newpage
\vspace{.25in}
\noindent {\bf Historical Note:}  The formal ideas for the use of cluster sampling were introduced in the same the pioneering paper by Jerzy Neyman in 1934 in which he advocated the use of optimal allocation for stratified sampling, and it was his combination of stratification and clustering this had such a profound influence on subsequent developments in the field of sampling and in large-scale survey practice.

\vspace{.25in}
\noindent {\bf Note on Proofs of Results:}  Cochran (1977) provides
formal derivations for some of the formulas presented here for cluster
sampling as well as for multi-stage cluster sampling and cluster
sampling when the clusters are of unequal size. Lohr (1999) also has
an excellent presentation on the relevant formulas and their derivation.


\section*{References:} 
Cochran, W. G. (1977). {\em Sampling Techniques.} 3rd. Edition. Wiley: New York.\\

\noindent Neyman, J. (1934). On the two different aspects of the representative
method: the method of stratified sampling and the method of purposive
selection (with discussion). {\em Journal of the Royal Statistical
  Society,} {\bf 97}, 558-625.\\

\noindent Henry, G. T. (1990). {\em Practical Sampling.} Sage: Newbury Park, CA.\\ 

\noindent Lohr, S. (1999). {\em Sampling: Design and Analysis.}
Duxbury: Pacific Grove, CA.\\

\noindent Williams, B. (1978). {\em A Sampler on Sampling.} Wiley: New York.

\end{document}

Statistics 36-203                                                                                                                   Winter, 1999



9

