# Fall 2019: Intermediate Statistics (36-705)

Instructor Information

**Instructor: **Siva Balakrishnan

**Email: ** siva@stat.cmu.edu

**Office Hours: **Mondays 1:30PM - 2:30PM

**Location: **BH 132K

TA Information

**TA: ** Aleksandr Podkopaev

**Email: ** apodkopa@andrew.cmu.edu

**Office Hours: ** TBA

**Location: ** TBA

**TA:** Ilmun Kim

**Email: ** ilmunk@andrew.cmu.edu

**Office Hours: ** TBA

**Location: ** TBA

Course Description

This course covers the fundamentals of theoretical statistics. Topics include: concentration of measure, basic empirical process theory, convergence, point and interval estimation, maximum likelihood, hypothesis testing, Bayesian inference, nonparametric statistics and bootstrap re- sampling. This course is excellent preparation for advanced work in Statistics and Machine Learning. See below for a detailed schedule.

Course Syllabus

The syllabus provides information on grading, class policies etc.

Lecture Notes

Lecture 1: (8/26) A very brief review
Lecture 2: (8/28) Concentration inequalities
Lecture 3: (8/30) More concentration inequalities
Lecture 4: (9/4) Even more concentration inequalities
Lecture 5: (9/6) Convergence of random variables
Lecture 6: (9/9) More Convergence and the CLT
Lecture 7: (9/11) CLT and variants
Lecture 8: (9/13) Uniform Laws
Lecture 9: (9/16) Applications of Uniform Convergence
Lecture 10: (9/18) Rademacher Complexity
Lecture 11: (9/23) Sufficient Statistics
Lecture 12: (9/25) Minimal Sufficiency
Lecture 13: (9/27) Exponential Families
Lecture 14: (9/30) Constructing Estimators
Lecture 15: (10/2) Cramer-Rao Lower Bound
Lecture 16: (10/4) Decision Theory Basics
Lecture 17: (10/7) Bounding the Minimax Risk
Lecture 18: (10/9) Consistency of the MLE
Lecture 19: (10/11) Asymptotic Normality of the MLE
Lecture 20: (10/14) Hypothesis Testing and the Neyman-Pearson Lemma
Lecture 21: (10/16) General Purpose Tests
Lecture 22: (10/21) LRT, Two-Sample Testing, Permutation Test
Lecture 23: (10/23) Multiple Testing
Lecture 24: (10/28) More Multiple Testing
Lecture 25: (10/30) Confidence Intervals
Lecture 26: (11/4) Confidence Intervals and the Bootstrap
Lecture 27: (11/6) More on the Bootstrap
Lecture 28: (11/8) Causal Inference
Lecture 29: (11/11) More Causal + Non-Parametric Regression
Lecture 30: (11/13) High-dimensional Statistics
Lecture 31: (11/15) Linear Regression
Lecture 32: (11/18) Non-parametric Regression
(11/20) Class Canceled
Lecture 33: (11/22) Bayesian Inference
(11/25)-(11/27)-(11/29) Thanksgiving Break
Lecture 34: (12/2) Model Selection
Lecture 35: (12/4) Distances Between Distributions
Lecture 36: (12/6) Fano's inequality and more distances

Assignments

Assignment 1: Due on 8/30 at 3pm.
Assignment 2: Due on 9/5 at 3pm.
Assignment 3: Due on 9/12 at 3pm.
Assignment 3 Source
Assignment 4: Due on 10/3 at 3pm.
Assignment 5: Due on 10/10 at 3pm.
Assignment 6: Due on 10/25 at 3pm.
Assignment 7: Due on 11/8 at 3pm.
Assignment 7 Source
Assignment 8: Due on 11/19 at 3pm.
Assignment 8 Source
Assignment 9: Due on 12/6 at 3pm.
Assignment 9 Source

Assignment solutions

Assignment 1
Assignment 2
Assignment 3
Assignment 4
Assignment 5
Assignment 6
Assignment 7
Assignment 8
Assignment 9

Practice Exam 1

Solutions to Practice Exam 1

Practice Exam 2

Solutions to Practice Exam 2

Practice Final (Ignore MCMC question, No Solutions this time)

Solutions to Midterm 1

Solutions to Midterm 2