\section*{Autocorrelation Function and Periodogram}
Last time we looked at the autocorrelation function (acf) which just
gave the correletion of a series with itself at various lags.  This
time we will also look at the {\bf periodogram}, an estimate of the
{\em spectrum\/} of the time series.

The periodogram is a summary description based on a representation of
an observed time series as a superposition of sinusoidal waves of
various frequencies.  The practical use of this is the empirical
observation that many times series show cyclic fluctuations, but the
underlying sin and cosine fucntions are not obvious.

To motivate the periodogram, consider a very simple model for a time
series $y_t$ exhibiting cyclic fluctuations with a known period, $p$.
We assume that the the series is
$$ y_t = \alpha cos(\omega t) + \beta sin(\omega t) + z_t $$ where
$z_t$ is a white noise sequence, $\omega = 2 \pi / p $ is the known
frequency of the cyclic fluctuations and $\alpha$, $\beta$ are
parameters that we shall estimate (by least squares).  (Note the
relationship between period $p$ and frequency $2 \pi / p$).

After some calculations ($\ldots$, I'll ignore the math), one finds
that the least squares estimates are
$$ \begin{array}{lcl} \hat{\alpha} & = & 2 \left \{ \sum_{t=1}^n cos(t
\omega) \right \} / n \\ \hat{\alpha} & = & 2 \left \{ \sum_{t=1}^n
sin(t \omega) \right \} / n
\end{array} $$

Second (random) fact.  The regression sum of squares (sum of square of
the fitted values) is
$$ n ( \hat{\alpha}^2 + \hat{\beta}^2 ) /2 $$ This is proportional to
the squared amplitude $\hat{\alpha}^2 + \hat{\beta}^2$ of the fitted
sinusoid.

That is a quite simplistic model.  More realistic would be if we not
only did not know the amplitude but also didn't know the frequency of
the series.

So consider a model of the form
$$ y_t = \sum_{k=1}^m \left \{ \alpha_k cos(\omega_k t) + \beta_k
sin(\omega_k t) \right \} + z_t$$

This is again an easy model to estimate, provided we pick the right
set of frequencies.  If we take $\omega_k = 2 \pi k / n $, with $k <
n/2$ then some nice things happen.  For starters this is the only set
of frequencies that we really can estimate (smaller periods would
jiggle more often than the data, longer periods would not see an
entire cycle in the data!).  This set of frequencies also represents a
set of orthogonal functions.  That is the correlation between any pair
of these funcions (measured at the data points) is zero.

The nice thing that happens (because of the orthogonality) is that the
estimates of $\hat{\alpha}$ and $\hat{\beta}$ are the same as before,
namely,
$$ \begin{array}{lcl} \hat{\alpha}_k & = & 2 \left \{ \sum_{t=1}^n
cos(t \omega_k) \right \} / n \\ \hat{\alpha}_k & = & 2 \left \{
\sum_{t=1}^n sin(t \omega_k) \right \} / n
\end{array} $$

Furthermore, $\hat{\alpha}_k^2 + \hat{\beta}_k^2$ again gives the sums
of squares associated with each of the $m$ fitted sinusoids.  Finally,
(via some math, or common sense), if all the frequencies are fit, then
the regression model fits exactly.

Some further mathematical manipulations show that
$$\hat{\alpha}^2 + \hat{\beta}^2 = 2 \left [ \left \{ \sum{y_t
  cos(\omega t)} \right \}^2 + \left \{ \sum{y_t sin(\omega t)} \right
\}^2 \right ] \ n $$

This is quite intuitive, the terms are just the inner product of the
data with the (basis) function.  This is what regression (and
particularly ANOVA) was all about.  

We can now define the ordinates of the periodogram function,
$I(\omega)$ for $ 0 <= \omega <= \pi $, by
$$ I(\omega) = \left [ \left \{ \sum{y_t cos(\omega t)} \right \}^2 +
\left \{ \sum{y_t sin(\omega t)} \right \}^2 \right ] \ n $$.

Thus we can ``parition'' the total sum of squares for the data $y_t$
as
$$ \sum_{t=1}^n = I(O) + 2 \sum_{j=1}^m I(2 \pi j/n) + I(\pi) $$
where $m$ is the largest integer less than $n/2$ and the final term
($I(\omega)$) is absent if n is odd.
The function represents that power (amount of energy) at each of the
fundamental frequencies that can be estimated.

A plot of $I(\omega)$ against $\omega$ is called the periodogram of
$y_t$. 

Here is an example.
\begin{verbatim}
> par(mfrow=c(4,1))
> tsplot(uk.male)
> tsplot(ts(Mod(fft(uk.male))[2:36]))
> title("Really raw periodogram")
> spectrum(uk.male)
> spectrum(uk.male,spans=c(3,5))
\end{verbatim}
\centerline{\psfig{figure=periodogram.eps,angle=-90,width=6in}}
Notice that the main power corresponds to a frequency of 12 months,
but there is also significant power at some other frequencies (like a
two-year cycle).  The graphs are a time-series plot of the raw data, a
(raw, unsmoothed) periodogram, an estimate of the spectrum (on a
10*log\_10) scale, and an even more smoothed spectral estimate.

Now remember the lutenizing hormone example.  We will look at the
late1 series from these data.
\begin{verbatim}
> tsplot(late1)
> tsplot(ts(Mod(fft(late1))[2:24]))  
> title("Really raw periodogram")  
> spectrum(late1)                  
> tsplot( ts(10^(spectrum(late1, plot=F)$spec/10)))
> title("Spectral estimate on natural scale")
\end{verbatim}
\centerline{\psfig{figure=periodogram2.eps,angle=-90,width=6in}}

% Local Variables: 
% mode: latex
% eval: (LaTeX-math-mode t)
% eval: (my-latex-mode)
% eval: (read-abbrev-file "~/.abbrev_defs")
% abbrev-mode: t
% TeX-master: "class"
% End: 
