# The Truth About Linear Regression

This is basically a compilation of the lecture notes I wrote when teaching
36-401, Modern Regression,
in fall 2015. I offer it here on the chance that it might be of interest
to those learning, or teaching, linear regression. There's no shortage of
resources on that, but I have tried to present the subject as though statistics
had made *some* progress since 1960, de-emphasizing bits of theory which
rely on Gaussian noise and correctly-specified linear models, in favor of
more computationally-intensive, but robust, techniques. If anything,
I did not go far enough.

The manuscript has some over-lap with
Advanced Data
Analysis from an Elementary Point of View (especially that book's
second chapter, "The Truth About Linear Regression"), but also a lot of new and
lower-level material. Comments and (especially) corrections are appreciated.

---Cosma Shalizi

#### Current outline

- Optimal Prediction
- Introducing Statistical Modeling
- Simple Linear Regression Models, with Hints at Their Estimation
- The Method of Least Squares for Simple Linear Regression
- The Method of Maximum Likelihood for Simple Linear Regression
- Diagnostics and Modifications for Simple Regression
- Inference on Parameters
- Predictive Inference for the Simple Linear Model
- Interpreting Parameters after Transformation
- F-Tests, R^2, and Other Distractions
- Simple Linear Regression in Matrix Format
- Multiple Linear Regression
- Diagnostics and Inference for Multiple Linear Regression
- Polynomial and Categorical Regression
- Multicollinearity
- Tests and Confidence Sets
- Interactions
- Outliers and Influential Points
- Model Selection
- Review
- Weighted and Generalized Least Squares
- Variable Selection
- Trees
- The Bootstrap I
- The Bootstrap II

(Last text update: typo corrections, 6 May 2024)