# Fall 2017: Intermediate Statistics (36-705)

Instructor Information

**Instructor: **Siva Balakrishnan

**Email: ** siva@stat.cmu.edu

**Office Hours: **Mondays 1:30PM - 2:30PM

**Location: **BH 132K

TA Information

**TA: ** Purvasha Chakravarti

**Email: ** pchakrav@andrew.cmu.edu

**Office Hours: ** Wednesday 6-7pm

**Location: ** BH 132Q

**TA:** Zongge Liu

**Email: ** zonggel@andrew.cmu.edu

**Office Hours: ** Tuesday 2-3pm

**Location: **BH 132A

**TA:** Shengming Luo

**Email: ** shengmingluo@cmu.edu

**Office Hours: ** Tuesday 6-7pm

**Location: **BH 132Q

**TA:** Daren Wang

**Email: ** darenw@andrew.cmu.edu

**Office Hours: ** Wednesday 1.30-2.30pm

**Location: ** BH 132Q

Course Description

This course covers the fundamentals of theoretical statistics. Topics include: concentration of measure, basic empirical process theory, convergence, point and interval estimation, maximum likelihood, hypothesis testing, Bayesian inference, nonparametric statistics and bootstrap re- sampling. This course is excellent preparation for advanced work in Statistics and Machine Learning. See below for a detailed schedule.

Course Syllabus

The syllabus provides information on grading, class policies etc.

Course Calendar

The calendar has an approximate week-by-week schedule. Consult this document to know
when the in-class exams are.

Lecture Notes

Lecture 1: (8/28) A very brief review
Lecture 2: (8/30) Concentration inequalities
Lecture 3: (9/1) Concentration inequalities
Lecture 4: (9/6) Convergence
Lecture 5: (9/8) Convergence
Lecture 6: (9/11) Central limit theorem
Lecture 7: (9/13) Uniform laws and empirical process theory
Lecture 8: (9/15) Uniform laws and empirical process theory
Lecture 9: (9/18) VC dimension
Lecture 10: (9/20) Rademacher complexity
Lecture 11: (9/25) Sufficiency
Lecture 12: (9/27) Minimal Sufficiency and Rao-Blackwell
Lecture 13: (9/29) Exponential Families
Lecture 14: (10/2) Parametric Estimation
Lecture 15: (10/4) Fisher Information and Decision Theory
Lecture 16: (10/6) Decision Theory
Lecture 17: (10/9) Minimax and Consistency of MLE
Lecture 18: (10/11) Asymptotic Normality of MLE
Lecture 19: (10/13) Hypothesis Testing
Lecture 20: (10/16) More Hypothesis Testing
Lecture 21: (10/18) LRT and Permutation Cut-offs
Lecture 22: (10/23) Multiple Testing
Lecture 23: (10/25) FDR and Confidence Sets
Lecture 24: (10/29) Confidence Intervals
Lecture 25: (11/1) Confidence Intervals and Causal Inference
Lecture 26: (11/3) Causal Inference
Lecture 27: (11/6) Non-parametric Regression
Lecture 28: (11/8) High-dimensional Statistics
Lecture 29: (11/13) More High-dimensional Statistics
Lecture 30: (11/15) Bayesian Inference
Lecture 31: (11/17) MCMC
Lecture 32: (11/27) MCMC and Bootstrap
Lecture 33: (11/29) Bootstrap and Model Selection
Lecture 34: (12/1) Model Selection
Lecture 35: (12/4) Distances between Distributions
Lecture 36: (12/6) (Completely Optional Lecture) Research in Theoretical Statistics/Statistical Machine Learning

Lecture Notes Source Files

Assignments

Assignment 1: Due on 8/31 at 3pm.
Assignment 2: Due on 9/7 at 3pm.
Assignment 3: Due on 9/14 at 3pm.
Assignment 4: Due on 10/5 at 3pm.
Assignment 5: Due on 10/13 at 3pm.
Assignment 6: Due on 10/19 at 3pm.
Assignment 7: Due on 11/10 at 3pm.
Assignment 8: Due on 11/27 at 3pm.
Assignment 9: Due on 12/8 at 3pm.

Assignment solutions

Assignment 1
Assignment 2
Assignment 3
Assignment 4
Assignment 5
Assignment 6
Assignment 7
Assignment 8
Assignment 9
Practice Exam 1
Practice Exam 2
Practice Final
Exam 1
Exam 2
Exam 1 Solutions
Exam 2 Solutions