\section*{Time Series}
I will be drawing my motivation and some of my lecture material from
the book {\em Time Series: A Biostatistical Introduction\/} by Peter
J. Diggle.  I'll also draw some material from the book {\em The
  Analysis of Time Series: An Introduction\/} by C. Chatfield, and
various other sources as needed.

Many time series books draw their motivation from economic and
finance, and much of the methodology is widely applied there.  However
the general ideas are useful in all areas of statistics from
Biostatistics (the approach I will follow), to all sorts of physical
phenomena (like ocean temperatures at different depths), to geology
(where I first met this material---analyzing the core samples from
exploratory drilling for nickel), to all sort of spatial problems
(which need very similar mathematics).  

The basic mathematical tools are some simple concepts about correlation,
some ideas from other stochastic processes (things like Markov
chains), and some Fourier theory.   We will have to develop some
mathematics, but we will try to motivate the work with examples.


\section*{What is a time series}

In its simplest form a {\em time-series\/} is a set of data
$ {y_t: t=1, \ldots , n}$ in which the subscript t indicates the time
at which the datum is observed.  That notation seems to imply that the
data points must be equally spaced in time.  While that is certainly
the most common case, it is not necessary. A more honest notation
might then be
$ {y(t_i): i=1, \ldots , n}$.  (Note that most software has
difficulties with non-equally spaced data!)

Those two notations cover the most common situations, but there are a
lot of bells and whistles to consider.  The data may have a
corresponding {\em predictor\/} series.  For example we might monitor
the progress, over time, of a fever in a baby.  The predictor series
might be the amount of medicine the child is given at varying times.
Another common extension is to consider multivariate time-series where
we measure multiple responses at the same time.  To continue in the
same vein, we might measure the temperature, pulse rate, and blood
pressure of the baby.  

What makes the data interesting (and difficult to analyze) is that it
is {\em NOT\/} reasonable to assume that the observations are iid.
At the very least we need to be willing to drop the {\em
  independent\/} part of the assumptions---while we might want to
retain the identically distributed.  In the above ``examples'' it
makes little sense to assume that the baby's temperature at one time
point is independent of the temperature at the next time.  If YOU were
the parent, the whole point of keeping records would be to discover
when the temperature is rising/falling, and you probably expect the
observations to be correlated; which implies they are not independent.


\subsection*{{Example 1}}

Levels of lutenizing hormone in blood samples.   Blood samples are
taken from a healthy woman every ten minutes over an eight hour
period.  The samples are assayed to determine the lutenizing hormone
(LH) levels.  The following data and plots show three series---one
from the early follicular phase of the subject's menstrual cycle, the
others from the late follicular phase of two successive cycles.  
There are clear cycles.  In principle, the phenomenon being
investigated is happening in real time, but we can't collect and assay
the data that way.  It makes a lot more sense to do equally spaced
intervals.
Here is the data.
\begin{verbatim}
     Lutenizing hormone.

     col 1 : serial number
     col 2 : Cycle 1, early follicular phase 1 (not discussed)
     col 3 : Cycle 1, late follicular phase
     col 4 : Cycle 2, early follicular phase
     col 5 : Cycle 2, late follicular phase

Serial Early1 Late1 Early2 Late2
   1  2.2  5.5  2.4  4.3
   2  2.2  4.5  2.4  4.6
   3  2.3  5.1  2.4  4.7
   4  2.0  5.5  2.2  4.1
   5  1.6  5.7  2.1  4.1
   6  1.4  5.1  1.5  5.2
   7  1.8  4.3  2.3  5.0
   8  2.2  4.8  2.3  4.4
   9  2.9  5.6  2.5  4.2
  10  2.6  5.9  2.0  5.1
  11  2.4  6.0  1.9  5.1
  12  2.1  5.1  1.7  4.7
  13  3.0  5.2  2.2  4.4
  14  2.5  4.4  1.8  3.9
  15  2.7  5.5  3.2  5.4
  16  2.2  5.4  3.2  5.9
  17  2.4  4.1  2.7  4.2
  18  2.7  4.4  2.2  4.1
  19  3.1  4.7  2.2  4.1
  20  2.5  4.6  1.9  3.6
  21  2.4  6.0  1.9  3.1
  22  2.3  5.6  1.8  4.8
  23  2.4  5.1  2.7  5.1
  24  1.9  4.7  3.0  5.1
  25  3.3  4.8  2.3  4.5
  26  3.8  5.5  2.0  4.6
  27  3.7  5.1  2.0  5.8
  28  3.5  5.2  2.9  5.0
  29  3.1  5.0  2.9  5.1
  30  2.7  4.0  2.7  4.5
  31  4.1  3.7  2.7  4.2
  32  4.0  4.8  2.3  6.0
  33  3.4  5.9  2.6  5.6
  34  3.2  5.5  2.4  5.4
  35  3.7  4.9  1.8  5.0
  36  3.6  4.4  1.7  4.4
  37  4.1  4.7  1.5  4.6
  38  2.0  4.2  1.4  5.7
  39  4.6  5.5  2.1  5.2
  40  4.1  4.9  3.3  5.0
  41  3.2  4.8  3.5  4.4
  42  2.9  4.5  3.5  5.7
  43  2.7  4.9  3.1  5.7
  44  3.0  4.9  2.6  4.8
  45  0.   4.5  2.1  3.4
  46  0.   4.2  3.4  5.5
  47  0.   4.9  3.0  5.5
  48  0.   5.9  2.9  5.6
\end{verbatim}

Here are some simple analyses in S.
\begin{verbatim}
> lh <- read.table("lh.dat", head=T)
> lh                                
   Serial Early1 Late1 Early2 Late2 
 1      1    2.2   5.5    2.4   4.3
 2      2    2.2   4.5    2.4   4.6
 3      3    2.3   5.1    2.4   4.7
  . . . . .
> early <- ts(lh$Early2)
> late1 <- ts(lh$Late1)
> late2 <- ts(lh$Late2)
summary(lh)
      Serial          Early1          Late1           Early2    
 Min.   : 1.00   Min.   :0.000   Min.   :3.700   Min.   :1.40  
 1st Qu.:12.75   1st Qu.:2.200   1st Qu.:4.575   1st Qu.:2.00  
 Median :24.50   Median :2.650   Median :4.900   Median :2.30  
 Mean   :24.50   Mean   :2.594   Mean   :4.990   Mean   :2.40  
 3rd Qu.:36.25   3rd Qu.:3.225   3rd Qu.:5.500   3rd Qu.:2.75  
 Max.   :48.00   Max.   :4.600   Max.   :6.000   Max.   :3.50  
      Late2      
 Min.   :3.100  
 1st Qu.:4.375  
 Median :4.800  
 Mean   :4.800  
 3rd Qu.:5.250  
 Max.   :6.000  
\end{verbatim}
I want to plot the data on the same scale, so I need the minimum (1.4)
and maximum (6.0).
\begin{verbatim}
> par(mfrow=c(2,2))
> tsplot(early,ylim=c(1.4,6.0))
> tsplot(late1,ylim=c(1.4,6.0))
> tsplot(late2,ylim=c(1.4,6.0))
\end{verbatim}

Notice the periodic nature of the plots.  Is this real periodicity, or
are we kidding ourselves.  Could we predict into the future?  The data
shows a cyclic nature, but the frequency of the cycle is probably not
known in advance.  Other data shows similar trends that are known in
advance.  These are often called seasonal trends.

As a digression, when you have multiple plots per page how do you get
to the next page?  I wrote the following function, which seems to do
the trick.  There might be a better way---I don't know it.
\begin{verbatim}
> newplot <- function() {
+ par(mfrow=par("mfrow"))}
\end{verbatim}

\centerline{\psfig{figure=lhplots.eps,angle=-90,width=6in}}

\subsection*{{Example 2}}

UK deaths from bronchitis, emphysema, and asthma.
The table and figure show the monthly returns of deaths in the UK from
bronchitis, emphysema, and asthma over the years 1974--1979,
inclusive.  Males and females are shown separately.  
\begin{center}
\begin{tabular}{r r r r r r r r r r}
\multicolumn{10}{c}{Males} \\\hline
  2134&  1863&  1877&  1877&  1492&  1249&  1280&  1131&  1209&  1492\\
  1621&  1846&  2103&  2137&  2153&  1833&  1403&  1288&  1186&  1133\\
  1053&  1347&  1545&  2066&  2020&  2750&  2283&  1479&  1189&  1160\\
  1113&   970&   999&  1208&  1467&  2059&  2240&  1634&  1722&  1801\\
  1246&  1162&  1087&  1013&   959&  1179&  1229&  1655&  2019&  2284\\
  1942&  1423&  1340&  1187&  1098&  1004&   970&  1140&  1110&  1812\\
  2263&  1820&  1846&  1531&  1215&  1075&  1056&   975&   940&  1081\\
  1294&  1341
\end{tabular}
\end{center}
\begin{center}
\begin{tabular}{r r r r r r r r r r}
\multicolumn{10}{c}{Females} \\\hline
   901&   689&   827&   677&   522&   406&   441&   393&   387&   582\\
   578&   666&   830&   752&   785&   664&   467&   438&   421&   412\\
   343&   440&   531&   771&   767&  1141&   896&   532&   447&   420\\
   376&   330&   357&   445&   546&   764&   862&   660&   663&   643\\
   502&   392&   411&   348&   387&   385&   411&   638&   796&   853\\
   737&   546&   530&   446&   431&   362&   387&   430&   425&   679\\
   821&   785&   727&   612&   478&   429&   405&   379&   393&   411\\
   487&   574 
\end{tabular}
\end{center}

We read in the data and do a plot
\begin{verbatim}
> uk.female <- scan("uk-female.dat")           
> uk.female <- matrix(uk.female,ncol=2,byrow=T)[,2]
> uk.female <- ts(uk.female, start=74.0,freq=12)
> uk.female
     Jan  Feb  Mar  Apr  May  Jun  Jul  Aug  Sep  Oct  Nov  Dec
74:  901  689  827  677  522  406  441  393  387  582  578  666
75:  830  752  785  664  467  438  421  412  343  440  531  771
76:  767 1141  896  532  447  420  376  330  357  445  546  764
77:  862  660  663  643  502  392  411  348  387  385  411  638
78:  796  853  737  546  530  446  431  362  387  430  425  679
79:  821  785  727  612  478  429  405  379  393  411  487  574
\end{verbatim}
(Similarly for the males).

Now notice that there is a strong seasonal nature to the plots; the
winter of 75--76 was particularly severe;  and the females have fewer
deaths than the males.

At one level there is an obvious relationship between the mal and
female returns, namely their common seasonal pattern.  Is there more
than that (i.e., once the seasonality is removed, is there still a
relationship?) 

We will look at other examples as we need them.

\centerline{\psfig{figure=ukplots.eps,angle=-90,width=6in}}

\section*{What are we trying to do?}

One aim is to provide a concise description of a historic series.  The
description may be just a short summary, or some graphs, or a quite
complicated model for the underlying stochastic process.  

If we are more ambitious, we might try to forecast future values of
the series.  This is particularly important for economic series.

An important aim in many medical examples is detecting a change in a
series.  This is also often a consideration in quality control
situations.


\section*{Some Notation}

Random variables will be upper case letters, and realized values will
be lower case, thus the random variable might be
$Y_t:t=1,\ldots,n$ 
and the observed values would be
$y_t:t=1,\ldots,n$.

The underlying time scale might be continuous or discrete. (For
example, my blood pressure could be monitored continuously, but the
CLOSING value of the Dow Jones index is discrete).  For most purposed
the granularity of the time scale does not matter.  However one must
also worry about the whether the series itself is discrete-valued or
continuous-valued.  We will not (seriously) consider discrete-valued
time series.  There are entire books on Markov chains, point processes
and the like.


\section*{Trend and Stationarity}

The concepts of trend, serial dependence, and stationarity are central
to our work in time-series data.  

Let ${Y(t)}$ be a random function of time.  That is at each time point
$t$, $Y(t)$ has some distribution.  

The (generally non-random function), $\mu(t) = \rm{E}[Y(t)]$ is called the
mean-values function, or trend of the series.

Secondly, the {\em autocovariance\/} function
$$
\gamma(s,t) = \rm{E}[{Y(t) - \mu(t)}{Y(s) - \mu(s)}] $$
describes how the series at time $t$ correlates with the series at
time $s$.  Clearly $\gamma(t,t)$ is the variance at time $t$.  

The series is considered to be stationary if the probabilistic
structure of ${Y(t)}$ is unaffected by time.  Loosely speaking, the
series looks about the same everywhere.  A formal definition of {\em
  strict stationarity\/} is that for any positive integer $m$ and
times $t_i, i=1,\ldots,m$, the joint distribution of
${Y(t_i+s):i=1,\ldots,m}$ does not depend on $s$.

Strict stationarity is a very strong assumption that is hard to check
and rarely needed.  Much more common (and what we shall use) is the
idea of second-order stationarity, namely: $\mu(t) $ must be a
constant and $\gamma(t,s)$ must depend only on $|t-s|$, not the
particular values of $t$ and $s$. 

The simplest stationary random sequence is {\em white noise\/}  This
is a sequence of mutually independent random variables, each with mean
zero and finite variance $\sigma^2$.  For such a sequence the
autocovariance function is
$$
\gamma(t,s) = \left \{ \begin{array}{l l}
                      \sigma^2 & \rm{if } t = s \\
                      0 & \rm{if } t \neq s \end{array} \right . $$

There are many versions of white noise, one standard one is where each
of the rv's a gaussian distribtion.  Here are a few realizations of
such a series.
\begin{verbatim}
> tsplot(ts(rnorm(200)))
> tsplot(ts(rnorm(200)))
> tsplot(ts(rnorm(200)))
> tsplot(ts(rnorm(200)))

> plot( 1:300/30, dgamma(1:300/30, 1.5))
> tsplot(ts(rgamma(200,1.5)))           
> tsplot(ts(rgamma(200,1.5)))
> tsplot(ts(rgamma(200,1.5)))
\end{verbatim}
\centerline{\psfig{figure=gausian.eps,angle=-90,width=5in}}
\centerline{\psfig{figure=gamma.eps,angle=-90,width=5in}}

% Local Variables: 
% mode: latex
% eval: (LaTeX-math-mode t)
% eval: (my-latex-mode)
% eval: (read-abbrev-file "~/.abbrev_defs")
% abbrev-mode: t
% TeX-master: "class"
% End: 
