\section*{Discrete---Categorical---Qualitative---Frequency---Data}


\subsection*{References}
\begin{itemize}
\item {\sc S.~E.~Fienberg}, {\em The Analysis of Cross-Classified
    Categorical Data}, MIT Press, 1980. (a.k.a., the Xmas-Tree book)
\item {\sc R.~L.~Plackett}, {\em The Analysis of Categorical Data},
  Second Edition, Griffin, 1981.
\item {\sc D.~R.~Cox}, {\em Analysis of Binary Data}, several
  editions, Chapman and Hall.
\item {\sc S.~Haberman}, {\em Analysis of Qualitative Data}, Volumes 1
  and 2, Academic Press.
\item {\sc Y.~M.~M.~Bishop, S.~E.~Fienberg, and P.~Holland}, {\em
    Discrete Multivariate Analysis}, MIT Press, 1975.
\item {\sc S.~Haberman}, {\em The Analysis of Frequency Data},
  University of Chicago Press, 1974.  This is a very difficult,
  theoretical book.
\item {\sl W.~N.~Venables and B.~D.~Ripley}, {\em Modern Applied
    Statistics with S-Plus}, Chapter 7 on Generalized Linear Models.
\item {\sc A.~Agresti}, Lost title, recent....
\end{itemize}

The basic data is a table (of list) of counts, i.e., we will consider
cases where the {\em response} is categorical {\em and\/} the
explanatory variables are also categorical (also known as factors).
The following table shows how this fits into the general scheme of
statistics, and in particular where logistic regression (which we will
talk about) and regression/ANOVA (which we have talked about) fit in.

\begin{center}
\begin{tabular}{l|l|l|l}
&\multicolumn{3}{c}{Explanatory Variables}\\\hline\hline
Response & Categorical & Continuous & Both \\\hline
Categorical & Contingency Tables & Logistic Regression & Loglinear Model\\\hline
Continuous & ANOVA/MANOVA & Linear Regression & ANACOVA \\\hline
Both & ?/Hard & ?/Hard & ?/Hard\\\hline
\end{tabular}
\end{center}


\subsection*{History}
This subject has a long history, perhaps even longer than the history
of regression.  There have been lots of famous disputes about
appropriate methods that have led to lots of interesting statistics.
Look at any of the books.


\subsection*{Motivation}

Consider the following table.  These data, as are many of the
examples, are stolen from the Xmas-Tree book.

The response variable (Heart Disease) is binary and the other
variables have been broken up into 4 categories each.  
\ \\
\begin{center}
\begin{tabular}{l| l| r r r r}
\hline
Coronary & Serum & \multicolumn{4}{c}{Systolic} \\
Heart & Cholesterol & \multicolumn{4}{c}{Blood Pressure (mm Hg)} \\
Disease & (mg/100 cc) & $<127$ &$127$--$146$ & $147$--$166$ & $167 +$\\
\hline
&$< 200$     &   2&   3&   3&   4\\
Present&$200$--$219$&   3&   2&   0&   3\\
&$220$--$259$&   8&  11&   6&   6\\
&$>= 260$    &   7&  12&  11&  11\\ \hline
&$< 200$     & 117& 121&  47&  22\\
Absent&$200$--$219$&  85&  98&  43&  20\\
&$220$--$259$& 119& 209&  68&  43\\
&$>= 260$    &  67&  99&  46&  33\\
\hline
\end{tabular}
\end{center}

\paragraph{Exercise}
Understand the data in this table---which are the most and least
risky groups.


\section*{One dimensional tables---Poisson and Multinomial
  Distributions}

Recall the properties of the Poisson and Binomial distributions.

The Poisson distribution with mean $\lambda$ has density
$$
{\rm Poi}(\lambda):  {\rm Pr}(R=r) = ({\rm e}^{- \lambda} \lambda^r)/r!
$$
and that $ {\rm\bf E}(R) = \lambda $ and ${\rm\bf Var}(R) = \lambda$.

The binomial distribution with parameter $p$, ${\rm Bi}(1,p)$ has
properties,
${\rm Pr}(R=1) = p $, ${\rm Pr}(R=0) = 1-p $, ${\rm\bf E}(R) = p$, and
$ {\rm\bf Var}(R) = p(1-p)$.

Now assume we have a response variable $R$ which can take on 5
possible values. The example I have in mind is a classification of the
fingerprints of the left first fingers of males.  Here is some sample
data (drawn from the book by Plackett).

\begin{center}
\begin{tabular}{r r r r r}
Arch & Small Loop & Large Loop & Composite & Whorl \\ \hline
41 & 139 & 53 & 28 & 68 \\
\end{tabular}
\end{center}

There are (at least) two ways I could have generated the above data.
\begin{enumerate}
\item I decided I wanted 329 observations, and then found them and
  classified them (i.e., the sample size was fixed).
\item I collection observations until I got tired (ran out of
  time/money/) and ended up with 329 people
\end{enumerate}

Either way, if I selected people at random we should be able to assume
that the individual observations are independent. The {\sc
  Independence} is important but very difficult to check.  The
following scenario is an example where independence may not be
appropriate.  Suppose we actually selected 100 families (an got 329
men in the sample).  If there is a correlation between (say) race an
finger print type, then our independence assumption may not be valid. 

What questions are of interest?  We may wish to know what is the
proportion of our population with (say) {\sc small loops}?  Do you
have any good estimates?  What makes {\em your\/} estimate good?


\subsection*{Sampling Schemes and Likelihoods}

Assume sampling method 1 (that is a fixed sample size), and
classification probabilities of $p_1$,$p_2$,$p_3$,$p_4$, and $p_5$,
naturally with the constraint that $\sum{p_i} = 1.$

Then the probability that I observe the data that I did (this is
called the likelihood) is
$$
{\rm P}(R_1=41,\ldots,R_5=68) = {{329}\choose{41,139,53,28,68}}
\cdot p_1^{41} \cdot p_2^{239} \cdot p_3^{53} \cdot p_4^{28} \cdot p_5^{68} 
$$
where ${{329}\choose{41,139,53,28,68}}$ is the multinomial coefficient
which is defined to be equal to
$$
{329!}\over{41! 139! 53! 28! 69!} $$
If we now ask what is the maximum likelihood estimate of, say $p_1$,
then it is an easy exercise to show that
$$ \hat{p_1} = {{41}\over{139}} $$
Similarly for the other $\hat{p_i}$.

What happens under sampling method 2, i.e., where the total number of
observations is a random variable?  The multinomial distribution is
not longer appropriate, but each of the counts can be viewed as one
observation on a Poisson random variable.  Let $m_1$--$M_5$ denote the
mean values of Poisson distributions.  If we assume that the data are
generated independently and the distribution of each category is
Poisson, then
\begin{eqnarray*}
{\rm P}(R_1=41,\ldots,R_5=68)& =& \frac{{\rm e}^{-m_1} m_1^{41}}{41!}
\frac{{\rm e}^{-m_2} m_2^{139}}{139!}
\frac{{\rm e}^{-m_3} m_3^{53}}{53!}
\frac{{\rm e}^{-m_4} m_4^{28}}{28!}
\frac{{\rm e}^{-m_5} m_5^{68}}{68!} \\
 & = & \exp{-(m_1+m_2+m_3+m_4+m_5)} \frac{\prod{ {m_i}^{r_i}}}{\prod r_i!}
\end{eqnarray*}
 
It is (should be) obvious that $\hat{m_i} = r_i$ and the MLE of the
proportion in each category remains the same.

It is a fact that unless we wish to make inferences about population
sizes, then it doesn't matter which sampling method we use.
Mathematically this means that,
$$
{\rm P_{Poi}}(R_1,\ldots,R_5 | R_1+R_2+R_3+R_4+R_5=329) =
{\rm P_{Mult}}(R_1,\ldots,R_5 ) $$

We haven't finished with this simple example yet.  We need an
estimate of $ {\rm Var}(\hat{p_i}) $ so that we have a scale of comparison.
Recall that $\hat{p_i} = \frac{r_i}{n}$ and note that in our two
sampling cases:
\begin{enumerate}
\item $$ {\rm Var}(R_i) = n p (1-p)$$ so
$$ \hat{{\rm Var}(\hat{p_i})} = \frac{\hat{p} (1-\hat{p})}{n}$$
\item ${\rm Var}(R_1) = m_1$, but $p_1 = \frac{R_1}{R_1 + \cdots +
  R_5}$.  It turns out that
$$ {\rm Var}(p_i) = \frac{( \frac{m_i}{R_+}) (1 -
  \frac{m_i}{R_+})}{R_+}$$
which is equivalent to the multinomial case---because of the
conditional distribution argument.
\end{enumerate}


\subsection*{Another Example}

We examine 3343 families with 4 or more children and find out how many
boys in the first 4 children.  The data are

\begin{center}
\begin{tabular}{r r r r r}
0 & 1 & 2 & 3 & 4 \\ \hline
183 & 789 & 1250 & 875 & 246 \\
\end{tabular}
\end{center}

So far we have not considered models.  What is an appropriate model
for these data?  If we assume (a little erroneously) that $ {\rm
  Pr}({\rm Boy}) = 0.5$, and that births within a family are
independence then we might generate the following probabilities,

\begin{center}
\begin{tabular}{r r r r r}
0 & 1 & 2 & 3 & 4 \\ \hline
${\frac{1}{2}}^4$ & $4{\frac{1}{2}}^4$ & 6 ${\frac{1}{2}}^4 $&$ 4
{\frac{1}{2}}^4 $& ${\frac{1}{2}}^4$ \\\hline
$\frac{1}{16} $&$ \frac{1}{4} $&$ \frac{3}{8} $&$ \frac{1}{4} $&$ \frac{1}{16} $\\
\end{tabular}
\end{center}

which would give us an expected table of counts of
\begin{center}
\begin{tabular}{r r r r r}
0 & 1 & 2 & 3 & 4 \\ \hline
208.9 & 835.75 & 1253.6 & 837.75 & 208.9 \\
\end{tabular}
\end{center}

and then we need to ask how well does the data fit the model.

One test of this is called Pearson's $ {\rm X}^2$ which is
\begin{eqnarray*}
{\rm X}^2 &=& \sum{ \frac{({\rm O}-{\rm E})^2}{{\rm E}}} \\
 &=& \frac{(183 - 208.9)^2}{208.9} + \cdots + \frac{(246 -
   208.9)^2}{208.9} \\
 & = & 14.27
\end{eqnarray*}

Asymptotically (i.e., as the sample size gets big) this is distributed
as a $\chi^2_4$ random variable.  Why 4 df? Note that
$\chi^2_4(0.05) = 9.49 $, and $\chi^2_4(0.01) = 13.28 $.


\section*{$2 \times 2$ Tables}


\paragraph{Example 1}
Linus Pauling (Nobel laureate and Vitamin C fame) reports the
following {\em double blind\/} data on French skiers and their
propensity to get colds.

\begin{center}
  \begin{tabular}{l| r| r| r}
    & Cold & No Cold & \\\hline
    Placebo & 31 & 109 & 140 \\ \hline
    Vit C (1 gm/day) & 17 & 122 & 139 \\\hline
  \end{tabular}
\end{center}

How was (could have) these data been collected?  Two groups were
selected, one was given treatment and the other the placebo.  The aim
was to compare the probability of getting a cold in each group.  In
other words we have two binomials, one for Placebo and one for Vitamin
C and we wish to compare responses.  Here getting a cold is the
{\em response\/} variable and the treatment is the explanatory
variable.

\paragraph{Example 2}
The following data is the cross-classification of Perch Height by
Perch Diameter of 164 lizards.

\begin{center}
  \begin{tabular}{r| r| r |}
 Height & \multicolumn{2}{c}{Diameter (in)} \\
 (feet) & $<= 4.0 $ & $> 4.0 $ \\ \hline
 $> 4.75$ & 32 & 11 \\ \hline
 $<=4.75 $ & 86 & 35 \\ \hline
  \end{tabular}
\end{center}

How were these data collected?  Is the Poisson or Multinomial
distribution appropriate?

What sort of question are appropriate for these data?
\begin{itemize}
\item Is the proportion at height 1 $>$ height 2?
\item Is the proportion at diameter 1 $>$ diameter 2?
\item Is height independent of diameter?
\end{itemize}

Whether we can answer these questions depends on how the data were
sampled?


\subsection*{Two Binomial---Skiers and Vitamin C}

We have two groups of skiers---those that were given Vitamin C and
those that were not.  We wish to ask the question:  Is the
probability of getting a cold the same in both groups?

If the data is,
\begin{center}
  \begin{tabular}{l| r| r| r}
    & Cold & No Cold & \\\hline
    Placebo & 31 & 109 & 140 \\ \hline
    Vit C (1 gm/day) & 17 & 122 & 139 \\\hline
  \end{tabular}
\end{center}

then a reasonable model for the data might be
\begin{center}
  \begin{tabular}{l| r| r| }
    & Cold & No Cold \\\hline
    Placebo & $p_1$ & $(1- p_1)$  \\ \hline
    Vit C (1 gm/day) & $p_2$ & $(1- p_2)$  \\ \hline
  \end{tabular}
\end{center}

and the question becomes:  Is $p_1 = p_2 = p$?

A good estimate of the common $p$ is $\frac{(31+17)}{279}$.  But how
do we test if this common value of $p$ adequately describes the data?

Consider the generic table of data,
\begin{center}
  \begin{tabular}{| r| r| r }
    \hline
    $x_{1,1}$ & $x_{1,2}$ & $n_1$ \\ \hline
    $x_{2,1}$ & $x_{2,2}$ & $n_2$  \\ \hline
  \end{tabular}
\end{center}


\paragraph{Method 1}

Note that $\hat{p_1} = \frac{x_{1,1}}{n_1} $ and
$\hat{p_2} = \frac{x_{2,1}}{n_2} $ and we wish to test if
$$
p_1 - p_2 = 0 $$

Note that ${\rm E}(\hat{p_1} - \hat{p_2}) = p_1 - p_2 $ and that
\begin{eqnarray*}
  {\rm Var}(\hat{p_1} - \hat{p_2}) & = & {\rm Var}(\hat{p_1}) + {\rm
    Var}(\hat{p_2}) \\
  & = & \frac{p_1 (1 - p_1)}{n_1} + \frac{p_2 (1 - p_2)}{n_2}
\end{eqnarray*}

If we assume that $p_1 = p_2$ then
$$ 
 {\rm Var}(\hat{p_1} - \hat{p_2}) = (\frac{1}{n_1} + \frac{1}{n_2} )
                                    (1-p) p
$$

If we further assume that $\hat{p_1} - \hat{p_2}$ is approximately
normal with mean 0 and variance $ {\rm Var}(\hat{p_1} - \hat{p_2})$,
then we can test the hypothesis with the statistic
$$
\frac{\hat{p_1} - \hat{p_2}}
             {\sqrt{(\frac{1}{n_1} + \frac{1}{n_2}) (1-\hat{p}) \hat{p}}}
 \sim {\rm N}(0,1) $$

For the skiers example, this is $ = 2.19 $, which is quite a large
number.

We had to assume approximate normality.  Numerous numerical (and
theoretical) studies have shown this is a quite valid assumption even
when the counts in each ``cell'' of the table become quite small.


\paragraph*{Method 2}

An alternative test for $ p_1 = p_2 $ can be obtained by using
Pearson's ${\rm X}^2$,
$$
{\rm X}^2 = \sum{ \frac{({\rm O}-{\rm E})^2}{{\rm E}}} $$
where O are the observed values and E are the expected values under
the model.

For our type of data the expected table is
\begin{center}
  \begin{tabular}{| r| r| }
    \hline
    $n_1 \hat{p}$ & $n_1(1- \hat{p})$\\ \hline
    $n_2 \hat{p}$ & $n_2(1- \hat{p})$\\ \hline
  \end{tabular}
\end{center}
which is equal to
\begin{center}
  \begin{tabular}{| r| r| }
    \hline
    24.1 & 115.9 \\ \hline
    23.9 & 115.1 \\ \hline
  \end{tabular}
\end{center}

and thus ${\rm X}^2 = 4.81 = (2.19)^2 $.  For this example, the ${\rm
  X}^2$ statistic is the square of a ${\rm N}(0,1)$---so its
distribution is $\chi^2_1$.  In general,  
${\rm X}^2$ is a useful statistics that is distributed as a 
$\chi^2_*$.   There is some uninteresting algebra which show the
equivalence of the two tests.

Why is it on 1 degree of freedom?   Note that the original data had
only 2 degrees of freedom because the sample sizes were predetermined.
We used the data to estimate one parameter, $p$, which left 1 degree
of freedom for the test.

This test for equality of Binomials is more generally referred to as a
test of {\sc Homogeneity of Proportions}.  {\em 
Note the importance of identifying the {\sc Response} and {\sc Explanatory}
variables}. 


\subsection*{Poisson or Multinomial---Lizards on Perches}

We have one group of lizards which are cross-classified according to
the perch height and diameter.  Sampling may have been either
multinomial (we knew how many lizards there were) or Poisson (we
didn't know).  Notice we have 2 {\sc Response} variables, and not
explanatory variables, so appropriate questions about the data concern
the interrelationships between the responses.

\begin{center}
  \begin{tabular}{r| r| r |}
 Height & \multicolumn{2}{c}{Diameter (in)} \\
 (feet) & $<= 4.0 $ & $> 4.0 $ \\ \hline
 $> 4.75$ & 32 & 11 \\ \hline
 $<=4.75 $ & 86 & 35 \\ \hline
  \end{tabular}
\end{center}

The data and probabilities of observation are,
Consider the generic table of data,

\begin{minipage}[t]{2.5in}
\begin{center}
  \begin{tabular}{| r| r| r }
    \hline
    $x_{1,1}$ & $x_{1,2}$ & $n_1$ \\ \hline
    $x_{2,1}$ & $x_{2,2}$ & $n_2$  \\ \hline
  \end{tabular}
\end{center}
\end{minipage} \ probabilities \
\begin{minipage}[t]{2.5in}
\begin{center}
  \begin{tabular}{| r| r| r }
    \hline
    $p_{1,1}$ & $p_{1,2}$ & $p_{1,+}$ \\ \hline
    $p_{2,1}$ & $p_{2,2}$ & $p_{2,+}$  \\ \hline
    $p_{+,1}$ & $p_{+,2}$ & $p_{+,+}$  \\ 
  \end{tabular}
\end{center}
\end{minipage}

We may like to ask questions like ``Is Diameter independent of
Height?''.  For independence we know that marginal probabilities need
to be multiplied.  If we take the marginal Height and Diameter
probabilities as given then the independence hypothesis say that

\begin{center}
$\hat{p} = $ 
  \begin{tabular}{| r| r|  }
    \hline
    $p_{+,1}p_{1,+}$ & $p_{+,2}p_{1,+}$ \\ \hline
    $p_{+,1}p_{2,+}$ & $p_{+,2}p_{2,+}$ \\ \hline
  \end{tabular}
\end{center}
should be the observed probabilities under independence.

In other words, the table of fitted values under the independence
model is

\begin{center}
  \begin{tabular}{| r| r|  }
    \hline
    $\frac{x_{+,1}x_{1,+}}{x_{+,+}}$ & $\frac{x_{+,2}x_{1,+}}{x_{+,+}}$ \\ \hline
    $\frac{x_{+,1}x_{2,+}}{x_{+,+}}$ & $\frac{x_{+,2}x_{2,+}}{x_{+,+}}$ \\ \hline
  \end{tabular}
 $ = $  
  \begin{tabular}{| r| r|  }
    \hline
    30.9 & 12.1 \\ \hline
    87.1 & 33.9 \\ \hline
  \end{tabular}
\end{center}

The value of ${\rm X}^2$ for these data is 0.18 which should be
compared to a $\chi^2_1$---not very large.

% Local Variables: 
% mode: latex
% eval: (LaTeX-math-mode t)
% eval: (read-abbrev-file "~/.abbrev_defs")
% abbrev-mode: t
% TeX-master: t
% TeX-master: t
% End: 
