Fall 2018: Intermediate Statistics (36-705)

Instructor Information

Instructor: Siva Balakrishnan
Email: siva@stat.cmu.edu
Office Hours: Mondays 1:30PM - 2:30PM
Location: BH 132K

TA Information

TA: Ilmun Kim
Email: ilmunk@andrew.cmu.edu
Office Hours: Wednesdays, 10:00am - 11:00am
Location: Baker Hall 132Q

TA: Boyan Duan
Email: boyand@andrew.cmu.edu
Office Hours: Wednesdays, 11:00am -- Noon
Location: Baker Hall 132Q

TA: Benjamin LeRoy
Email: bpleroy@andrew.cmu.edu
Office Hours: Tuesdays, 1:30pm -- 2:30pm
Location: Baker Hall 232M


Course Description

This course covers the fundamentals of theoretical statistics. Topics include: concentration of measure, basic empirical process theory, convergence, point and interval estimation, maximum likelihood, hypothesis testing, Bayesian inference, nonparametric statistics and bootstrap re- sampling. This course is excellent preparation for advanced work in Statistics and Machine Learning. See below for a detailed schedule.

Course Syllabus

The syllabus provides information on grading, class policies etc.

Course Calendar

The calendar has an approximate week-by-week schedule. Consult this document to know when the in-class exams are.

Lecture Notes

  • Lecture 1: (8/27) A very brief review
  • Lecture 2: (8/29) Concentration inequalities
  • Lecture 3: (8/31) Concentration inequalities
  • Lecture 4: (9/5) Convergence of Random Variables
  • Lecture 5: (9/7) More Convergence
  • Lecture 6: (9/10) More Convergence
  • Lecture 7: (9/12) Central Limit Theorem
  • Lecture 8: (9/14) Uniform Laws
  • Lecture 9: (9/17) VC dimension
  • Lecture 10: (9/19) Bounding the Rademacher Complexity
  • Lecture 11: (9/24) Sufficient Statistics
  • Lecture 12: (9/26) Minimal Sufficiency
  • Lecture 13: (9/28) More Minimal Sufficiency and Exponential Families
  • Lecture 14: (10/1) Continuing Exponential Families
  • Lecture 15: (10/3) Constructing Point Estimators
  • Lecture 16: (10/5) MSE and Unbiased Estimators
  • Lecture 17: (10/8) Comparing Estimators and Decision Theory
  • Lecture 18: (10/10) Bayes and Minimax Estimators
  • Lecture 19: (10/12) Consistency and Inconsistency of the MLE
  • Lecture 20: (10/15) Asymptotic Normality of the MLE
  • Lecture 21: (10/17) Hypothesis Testing and Neyman Pearson Lemma
  • Lecture 22: (10/22) General Purpose Tests
  • Lecture 23: (10/24) Goodness-of-fit, Two-Sample and Permutation Tests
  • Lecture 24: (10/29) Multiple Testing (Guest Lecture: Aaditya Ramdas)
  • Lecture 25: (10/31) Multiple Testing (Guest Lecture: Aaditya Ramdas)
  • Lecture 26: (11/5) Confidence Sets
  • Lecture 27: (11/7) Conf. Sets and High-dimensional Mean Estimation
  • Lecture 28: (11/9) High-dimensional Mean Estimation and Low Dimensional Regression
  • Lecture 29: (11/12) LASSO and Non-parametric Regression
  • Lecture 30: (11/14) Non-Parametric Regression and Causal Inference
  • Lecture 31: (11/16) Causal Inference
  • Lecture 32: (11/26) Model Selection
  • Lecture 33: (11/28) Bayesian Inference
  • Lecture 34: (11/30) More Bayesian Inference + MCMC
  • Lecture 35: (12/3) MCMC + Bootstrap
  • Lecture 36: (12/5) Bootstrap
  • Lecture 37: (12/7) Distances Between Distributions (not on exam)

  • Last Years Lecture Notes

    Assignments

  • Assignment 1: Due on 8/30 at 3pm.
  • Assignment 2: Due on 9/6 at 3pm.
  • Assignment 3: Due on 9/13 at 3pm.
  • Assignment 4: Due on 10/5 at Midnight.
  • Assignment 5: Due on 10/19 at Midnight.
  • Assignment 6: Due on 10/31 at Midnight.
  • Assignment 7: Due on 11/20 at Midnight.
  • Assignment 8: Due on 12/6 at Midnight.

  • Assignment solutions

  • Assignment 1
  • Assignment 2
  • Assignment 3
  • Assignment 4
  • Assignment 5
  • Assignment 6
  • Assignment 7
  • Assignment 8


  • Practice Test 1

    Solutions to Practice Test 1
    Solutions to Test 1
    Practice Test 2

    Solutions to Practice Test 2
    Solutions to Test 2
    Practice Final
    Last Year's Final