Advanced Data Analysis II

36-402 Spring 1996

Course Policies and Syllabus

Instructor  Office  Telephone  e-mail  Office Hours
Mike Meyer  BH 229G  x8-3108 mikem@stat.cmu.edu After class, or just drop by.
Lectures T Th 9:00-10:20 PH A18B

Prerequisites: I will assume that you understand confidence intervals, hypothesis testing, basic distributions such as the normal, t, χ^{2} and F distributions, and that you have (more or less) mastered the material in 36-401, especially the material about regression models and diagnostics.

I expect a strong desire to analyze data and a willingness to be an active participant in the class. I will try to find interesting data analysis projects for each of you to work on, and will then encourage you to talk about them. You will have to use several computer packages during the semester, and that means using the campus Unix systems. You can probably do everything in either SAS or Splus, but I encourage you to learn both packages. There will be at least one assignment that requires you to use SAS.

Textbook: Modern Applied Statistics with S-Plus by W.N. Venables and B.D. Ripley, 1994, Springer

Other Useful Books: Statistical Models in S by J.M. Chambers and T. Hastie, 1992, Wadsworth. Statistics for Experimenters by George E.P. Box, William J. Hunter, and J. Stuart Hunter, 1978, Wiley (A good book for Anova).

Probable Topics We have several choices of material to cover. I’m open to suggestions, but I clearly have my own preferences.

1. ANOVA and designed experiments. In many ways this is just a continuation and special case of multiple linear regression, but there is some new language to learn and the approach to the problems is sometimes different. We will do this, for at least 4, and maybe more, weeks.

2. Categorial Data Analysis, including logistic regression, and the so-called generalized linear models. This covers the analysis of data with binary responses, or counts as the response variable. If we do this, it will take between 3 and 5 weeks.

3. Robust statistical methods and non-linear regression. This depends on your interests. It should take a couple of weeks.

4. Modern regression methods, including additive models and projection-pursuit. Depending on how carefully we do this, it can take from 2 to 5 weeks.

5. Continuous multivariate analysis. Data where there are multiple continuous-valued responses. This can be useful, but it will take many weeks to do well. Possible topics include, Discriminant Analysis, Principal Components, Factor Analysis (if I can figure it out), Multivariate ANOVA, Clustering Methods, Multivariate Graphics.

6. Survival Analysis. How to look at a clinical trial and try to understand, say, how effective a drug is. Possible topics include Survival Analysis and Hazard Rates, Censoring, Parametric and Non-parametric models, Goodness of Fit.

7. Time Series. Data recorded in time where an important concern is understanding the serial correlation of the data. We might explore both the time and frequency domain.

Left to my own devices, I’ll cover ANOVA, generalized linear models (with an emphasis on categorical data), and perhaps some survival analysis. Speak up with your opinion.

Class Organization I intend to give formal lectures but I will try to talk about (your) casestudies as often as possible. I usually hand out class notes and handouts where possible.

Overview This course will emphasize the conceptual basis of several data analytical techniques. We will not be overly concerned with computational or mathematical details, but nor will we shy from difficult theory when it is needed. I will stress the fact that these techniques, like all statistical techniques, involve assumptions. A typical statistical analysis begins with simple graphical and descriptive analyses followed by the application of formal statistical techniques. The analysis must then be followed up by diagnostics that check whether the assumptions have been violated. Most good analyses include a heavy dose of graphical techniques. The golden rule in this course is: Never do a statistical analysis without first plotting the data. Also bear in mind that there is never one right way to do an analysis—typically, many analyses are performed.

Homework. There will be approximately weekly assignments and a project. I encourage you to work together on the assignments but the work that you hand in should be your own. The assignments will make use of the computer packages. I will teach you the minimum you need to know about these computer packages—enough to do the work, but not enough to think yourself an expert. Statistical computing packages are very easy to use.

Late homework will not be accepted. You want your homeworks back quickly. The only way I can do that is if the homework is turned in on time.

NOTE: Your assignments should discuss the results of your analyses. Your conclusions and interpretation of the results are the most important part of your write-up. Never hand in raw computer output. Cut out plots, tables etc. from the output and include them in your report as needed. Include a short summary that someone not versed in statistics could read.

Assessment

Homework:  50%
Project:  25%
Take-home final: 25%