# Fall 2018: Intermediate Statistics (36-705)

Instructor Information

**Instructor: **Siva Balakrishnan

**Email: ** siva@stat.cmu.edu

**Office Hours: **Mondays 1:30PM - 2:30PM

**Location: **BH 132K

TA Information

**TA: ** Ilmun Kim

**Email: ** ilmunk@andrew.cmu.edu

**Office Hours: ** Wednesdays, 10:00am - 11:00am

**Location: ** Baker Hall 132Q

**TA:** Boyan Duan

**Email: ** boyand@andrew.cmu.edu

**Office Hours: ** Wednesdays, 11:00am -- Noon

**Location: ** Baker Hall 132Q

**TA:** Benjamin LeRoy

**Email: ** bpleroy@andrew.cmu.edu

**Office Hours: ** Tuesdays, 1:30pm -- 2:30pm

**Location: ** Baker Hall 232M

Course Description

This course covers the fundamentals of theoretical statistics. Topics include: concentration of measure, basic empirical process theory, convergence, point and interval estimation, maximum likelihood, hypothesis testing, Bayesian inference, nonparametric statistics and bootstrap re- sampling. This course is excellent preparation for advanced work in Statistics and Machine Learning. See below for a detailed schedule.

Course Syllabus

The syllabus provides information on grading, class policies etc.

Course Calendar

The calendar has an approximate week-by-week schedule. Consult this document to know
when the in-class exams are.

Lecture Notes

Lecture 1: (8/27) A very brief review
Lecture 2: (8/29) Concentration inequalities
Lecture 3: (8/31) Concentration inequalities
Lecture 4: (9/5) Convergence of Random Variables
Lecture 5: (9/7) More Convergence
Lecture 6: (9/10) More Convergence
Lecture 7: (9/12) Central Limit Theorem
Lecture 8: (9/14) Uniform Laws
Lecture 9: (9/17) VC dimension
Lecture 10: (9/19) Bounding the Rademacher Complexity
Lecture 11: (9/24) Sufficient Statistics
Lecture 12: (9/26) Minimal Sufficiency
Lecture 13: (9/28) More Minimal Sufficiency and Exponential Families
Lecture 14: (10/1) Continuing Exponential Families
Lecture 15: (10/3) Constructing Point Estimators
Lecture 16: (10/5) MSE and Unbiased Estimators
Lecture 17: (10/8) Comparing Estimators and Decision Theory
Lecture 18: (10/10) Bayes and Minimax Estimators
Lecture 19: (10/12) Consistency and Inconsistency of the MLE
Lecture 20: (10/15) Asymptotic Normality of the MLE
Lecture 21: (10/17) Hypothesis Testing and Neyman Pearson Lemma
Lecture 22: (10/22) General Purpose Tests
Lecture 23: (10/24) Goodness-of-fit, Two-Sample and Permutation Tests
Lecture 24: (10/29) Multiple Testing (Guest Lecture: Aaditya Ramdas)
Lecture 25: (10/31) Multiple Testing (Guest Lecture: Aaditya Ramdas)
Lecture 26: (11/5) Confidence Sets
Lecture 27: (11/7) Conf. Sets and High-dimensional Mean Estimation
Lecture 28: (11/9) High-dimensional Mean Estimation and Low Dimensional Regression
Lecture 29: (11/12) LASSO and Non-parametric Regression
Lecture 30: (11/14) Non-Parametric Regression and Causal Inference
Lecture 31: (11/16) Causal Inference
Lecture 32: (11/26) Model Selection
Lecture 33: (11/28) Bayesian Inference
Lecture 34: (11/30) More Bayesian Inference + MCMC
Lecture 35: (12/3) MCMC + Bootstrap
Lecture 36: (12/5) Bootstrap
Lecture 37: (12/7) Distances Between Distributions (not on exam)

Last Years Lecture Notes

Assignments

Assignment 1: Due on 8/30 at 3pm.
Assignment 2: Due on 9/6 at 3pm.
Assignment 3: Due on 9/13 at 3pm.
Assignment 4: Due on 10/5 at Midnight.
Assignment 5: Due on 10/19 at Midnight.
Assignment 6: Due on 10/31 at Midnight.
Assignment 7: Due on 11/20 at Midnight.
Assignment 8: Due on 12/6 at Midnight.

Assignment solutions

Assignment 1
Assignment 2
Assignment 3
Assignment 4
Assignment 5
Assignment 6
Assignment 7
Assignment 8

Practice Test 1

Solutions to Practice Test 1

Solutions to Test 1

Practice Test 2

Solutions to Practice Test 2

Solutions to Test 2

Practice Final

Last Year's Final